Well, you'd always colocate enough servers so that you can lose at least one machine without caring. E.g. HAProxy to 2 web tier machines on the back end. HAProxy will fail over to just one server no problem.
(And yes, you can heartbeat so you have two cheap physical HAProxy machines, too. This gets into sub-blade territory, where the 1U server is internally two or more complete low-ish power servers with independent power supplies, etc. )
That's the whole premise behind FOSS, you don't need to worry about all the licenses, and the hardware is so cheap it is effectively free and getting freer by the second, so you throw a lot of cheap hardware at the problem.
But agreed, "lot" in this case means at least two so one can fail and you don't need to care.
I guess my bigger point though, would be that when using something like AWS, I don't have to think/spend time on a lot of implementation details. When using colocation services/hardware/failover, I'm just adding a bunch of little things to my daily tasks and responsibilities. Sometimes this is a big deal (like in a bootstrapped two man team) and other times it's not.
Ultimately, I think it comes down to priority instead of possibility. If your company lives and dies on having reliable servers, you should probably roll your own. But if servers are 'just' a technical detail to your overall business model, then a cloud solution can be well worth the additional cost
When you're using something like AWS, you absolutely need to think about implementation details. As we've learned a few times now, sometimes AWS has datacenter-wide outages. You need to stripe across multiple availability zones, keep off-site backups, etc.
So yeah, you gotta think about it. A lot of the time, public cloud is the correct solution; however, you should have a solid understanding of what you need to do to run reliably in that cloud, how to build redundancy in the cloud you pick, when you might need to move to a different solution, and how to make those processes easier.
I don't think I said that you don't have to worry about implementation details at all, just that a cloud based solution like Amazon is often many orders of magnitude simpler than building/maintaining/repairing/replacing/updating physical machines at a co-location center.
We could argue about how many orders of magnitude, but I agree. It's absolutely easier in some ways. I'm just saying that it's easy to fall into the trap of thinking that EC2 (or whoever) is abstracted away from hardware failure/geographic problems when it definitely isn't.
http://arstechnica.com/business/2011/09/google-devops-and-di... is a bit overdramatized but the final section is a great summary of real stuff you need to think about on EC2, or any other provider. Again, I know you're not minimizing these issues, but some people certainly do.
(And yes, you can heartbeat so you have two cheap physical HAProxy machines, too. This gets into sub-blade territory, where the 1U server is internally two or more complete low-ish power servers with independent power supplies, etc. )
That's the whole premise behind FOSS, you don't need to worry about all the licenses, and the hardware is so cheap it is effectively free and getting freer by the second, so you throw a lot of cheap hardware at the problem.
But agreed, "lot" in this case means at least two so one can fail and you don't need to care.