I restored my iPhone and told it
not to restore the backup. It seems to be working better now. Remote is working and I might even get myself an AirPort extreme with airtunes now
Oh, I have re-setup my old Airport Extreme base station and set the phone to use that wifi, so I am not sure if that is why remote is now working, I will try it via my homehub later.