Let's talk about borrowed decks for a moment. So many people pass up on wonderful card magic, simply because it requires a gimmick, or preparation beforehand. I think this may be partly due to all the books and DVDs that boast that their material can be done "with any borrowed deck". New magicians see this and begin to imagine that every time they start to perform card magic, someone is going to stop them and thrust a borrowed deck into their hand. There's also the possibility that the magician wants to work with a borrowed deck just to prove that his skill is genuine. Well guess what? Most people who enjoy watching magic don't care. They want to see amazing things and some of the most amazing things aren't what we think of as impromptu. On that note, I would like to challenge your definition of impromptu.
If I have a favorite gimmick that I always carry with me everywhere I go, ready to perform with at a moment's notice, couldn't that be considered impromptu? When I get dressed in the morning, it goes like this: pants, shirt, Raven, Loops, TT, then my wallet goes in one hip pocket, my money, keys and a pack of bikes go in the other. The bikes, obviously, are a common item, but I would consider the other three gimmicks. Yet I'm ready to perform with them at any time, and reset is automatic, so when I'm done I'm already prepared to go again, whenever. I think this could be considered impromptu. Obviously some gimmicks require a setup before each performance, so they could never be impromptu, but certain tools shouldn't be dismissed just because there's preparation required.
Another aspect of magic that is overlooked is theory and principles of magic. Serious study of these will prepare you for truly impromptu performance. I was seven when I first became interested in magic, but I didn't know how to go about learning. I didn't know about magic shops, if there even were any in the town I lived in at the time, and my parents and teachers had no interest in helping me. Fortunately I was an advanced reader, I knew how to use the index in the school library and my parents believed any kind of reading was beneficial and would therefore buy me any book I asked for. So for the first five years of my magic career, I was immersed in theory. I had only every day objects to work with, so I studied, and practiced and became the MacGyver of magic. By the age of nine I could toss an object into the air and make it vanish before my friends' eyes. By ten I could make the object reappear in the most unlikely places. By eleven I could transform that item into something completely different in midair, and the new item could be passed out for examination. Finally at the age of twelve my father realized I wasn't going to quit, and bought me my first magic kit, and I was introduced to gimmicks for the first time. With my strong background in theory and technique, my new toys only made me more powerful.
To sum up, impromptu magic, in my opinion, is any magic that can be performed at a moment's notice. Technique can make you a good magician, but without studying the more mental side of magic you'll never be a great one. Finally, if you dismiss a tool just because you don't want to work with gimmicks, you're really limiting how great a magician you can possibly become. Don't become paranoid about your audience's suspicions. No matter how good you are, they know it's just a trick. They just want to be entertained.