Let's say you have two or three 5 minute sets for your restaurant job. How often would you change up and come up with different sets? Basically I'm trying to figure out how to deal with repeat customers if you get the same people coming every week to see your show.
You can deal with this from the very beginning by reserving one or more of your initial sets for repeat customers. In other words, of your initial five sets (or whatever you start out with), set aside one or two of them for repeat customers and perform the other sets as your regular material. The only thing preventing that is your willpower. I usually go into a walkaround gig with at least five sets of material; two or three will be the workers for the gig, while the rest will be reserved for repeat visits.
In the beginning, though, don't worry about too much about the need for more material. Three sets is enough to start; pay attention to how much repeat busness you actually get and adjust your repertoire accordingly.
Note that most professional working magicians are not constantly learning large volumes of new material. They develop acts and stick with them for extended periods of time. If you constantly learn new material, you risk not actually mastering any of it.
On one hand, it's easy for me to say that because I've been doing this for over 30 years and have accumulated a lot of material, so it isn't difficult for me to pull out something "fresh" for a spectator who has seen it all; on the other hand, though, my core walkaround repertoire is still made up of pretty much the same eight or ten tricks as when I started out. If it ain't broke...