I have all the sysadmin skills needed to run dedicated servers and in fact I was running and configuring most of our development and production systems for years. We did switch to Heroku for most of development and production projects lately and the difference in "time spent doing stupid IT" was enormous. literally tens of hours per month that I'm not spending now on starting, stopping, configuring, figuring out why the puppet install script fails thought it was working perfectly the last time, monitoring, etc, etc, etc. HUGE gains in productivity and totally worth the extra cost.
It all depends on how much you value your time. We are a consulting web dev house, so figuring out the price per hours is easy - its just what we bill our clients; but even for a regular startup it should be fairly straight forward. And this doesn't even take into account the opportunity cost. The cost of not doing something directly relevant for your startup outcome while you are fiddling with the servers.
All our development and staging systems are essentially free, mongolab/Mongohq have free plans that are enough for dev. That being said I'm definitely not looking to save 20$ as I bill 110€/hour so one saved hour I can devote to actuall work pays for half a year of payed heroku dyno;)
It all depends on how much you value your time. We are a consulting web dev house, so figuring out the price per hours is easy - its just what we bill our clients; but even for a regular startup it should be fairly straight forward. And this doesn't even take into account the opportunity cost. The cost of not doing something directly relevant for your startup outcome while you are fiddling with the servers.