Also not a restaurant worker, but here is my view anyway
At a buffet I could see that walk around might be distracting, since people are eating. But from the owners point of view it could work out. Eat slow (or get distracted watching the magician) and fill up faster. One of the tips on weight control is eating slower so you get full.
In a buffet setting what about more of a stage show? Have an empty area and do a small magic set? I have seen signs locally at a pizza buffet and a Golden Corral both advertising a Friday night magician. I have not been to either show so I don't know the setup of the show, but at least other buffets do shows.