I much prefer the middle ground of dedicated servers over the hassle of colocation and hardware management. (16GB RAM quad core with 2TB raid1 from OVH for £65 a month are a good affordable level, and very quick and easy to spin up new ones).
If bang for your buck is a concern, you might be interested in this offering from Hetzner: http://www.hetzner.de/en/hosting/produkte_rootserver/ex4s - you get double the RAM and a bit more disk space for a little bit less. I only have a month's worth of experience to talk about it, but other than the fact that it's in Germany (timezones) it's been great.
Yep, have used Hetzner servers too for some things and would use again, but for our customers being 90% in the US, and OVH offer a canadian location (and we may need some EU servers in the future).
I was also gonna come in here and mention the middle ground. Amazon is great either if you don't want to commit to anything, or if you have very spiky load and sometimes need to spin up a lot of servers.
Building your own servers is dirt cheap, but there's always the risk of hardware breaking, and you can't exactly scale up very quickly.
But the middle ground, rented dedicated servers, can be a perfect compromise for a lot of companies. Scaling up and adding servers might take a day instead of instantly, and it costs more than owning servers, but you get all the benefits of managed hosting, broken hardware simply isn't something you need to worry about anymore. If a server goes wonky, just scrap it and rent a new one.
