I have received countless "challenges" in the form of comments about a trick. these people just see magic as an attack to their self ego; they do not want to be "folled" they want to be the ones that know it all.
you could go with your line of work that you mentioned. But just be aware that by you responding they already got you---you do not want to pay ttention to them. if someone says something pretend they aren'te ven there and continue your effect. more often than not the rest of the audience will ignore them.
But for those extra rowdy guys I like to change things out as they are getting loud. Example on the ACR routine, they say that the card is still on top....you then stop..look at them and say " well we have a doubter here, come on here!" you then cut the deck, ask him to place the cad himself in the middle..and then ask him...you your self placed that card in the middle, you did not see any tricks from my side, correct?---as I give that line I did a classic pass and voila the card is on top

. " oo...gosh did you see that? maybe it was too fast for you, let me do it again.." and you can then repeat it with a Hoffmann pass. I usually ahdn out the deck immediately to the heclker and ask him to take the top card...and bang!!

this more often than not take care of even the raunchiest doubters amd quashes tehm by basically embarrasing them and erasing all theories.
or you could go right into a 2 card monte...this one will not only baffle all spectators, it will even create shock to the smart alleck.
just a few pointers that I have been using for years.