I second that, fallingblood. Here's my in-depth analysis on why that is a bad line.
The line reinforces that the secret of the trick is important, whereas the job of the magician is to try and make the audience care as little about the secret as possible, and accept the trick at face value. It is only when the audience looks at a trick in this way that it becomes truly magical. That line brings them down to earth and strengthens the idea that what you did was not magical but mere trickery.
When someone treats a magic trick as a puzzle, they expect an answer (the secret) to be provided for them, and they will end up resenting the magician when one is not given.
The way the line is delivered is often in a mocking "I know the secret and you don't" tone. All this does is distance the magician and spectator and allows them to identify less with you. This opens you up for heckling and insults.
There are a number of other,
more mature ways of responding to that question, but the more advanced magician will realise that the best thing is to learn how not to be asked the question in the first place.