Have a quick check of this link:
Magic Shops in East Anglia. That's courtesy of Bananafish, and the Ipswich Magical Society.
Also, I know Gt Yarmouth quite well, having worked there for two years in the past. There's no magic shop as such, but take a walk down 'Regent Road/Street' (aka 'Tacky Street' to locals) and you'll find all the charming

seaside-y shops that sell all kinds of stuff. One does stock magic, but it's more the practical joke kind of magic.
You'll also find no end of items that have the potential to be cheap props - I'm thinking cuddly white rabbits, jazzy storage boxes and other everyday things.
I remember seeing a stall with a magician demonstrating his wares - he had some good stuff, but it was all overpriced as he was obviously praying on tourists. I think his Bike decks were £3 or something ridiculous.
Are you off to a holiday park in the Yarmouth area? There may well be a magician doing a cabaret there... I know a few that do the Yarmouth circuit. If you are going to a holiday park, then call them and find out who their cabarets are for the days you are staying. Let us know who they are and I'll tell you what to expect. Those that are billed as comedy may well do a bit of magic too.
Most of them are BRILLIANT.