I've been using Siri for years and I get the trepidation, but it's really not awkward to do voice commands around people. In social situations, I look at it as simultaneously letting people around me know what I'm doing with the phone while I do it. Instead of saying "OK, let's see a movie. Here, I'll open Fandango and see what's playing around here." (or just pulling up the phone without telling them, which I'd regard as weirder, personally), I say "Yeah a movie sounds fun 'What movies are playing nearby?'" and we're immediately looking at movie times together. YMMV, but I find breaking eye contact for 15 seconds a lot more socially intrusive than a 2 second voice command.
I also find Siri works find with a low voice that nobody notices if I'm around people but not socializing (eg the open office). It just sounds like I'm taking a quick, quiet call at most.
The situation you're describing is one where the other people around you care what you're doing. In the vast majority of situations that isn't the case. The guy sitting next to me at work does not want to hear about me planning to see a movie later on tonight.
I also find Siri works find with a low voice that nobody notices if I'm around people but not socializing (eg the open office). It just sounds like I'm taking a quick, quiet call at most.