I've been using nginx for a long time now, and it's been great. Currently, I host my rails app combining nginx and unicorn with bluepill to manage the processes. I just don't trust passenger. Another specialty is that I use UNIX sockets instead of tcp ports for the reverse proxy, something unique to nginx/unicorn I think.
The most compelling use case I've seen where Passenger loses to Unicorn is rolling restarts. The biggest trust issue that I've seen is that of source editability - Unicorn and Mongrel are easier to debug and patch, while Passenger requires a commanding knowledge of Apache to fix.