i've known this secret for a while.
i went from paying $4k+/month spanning over 10+ servers in amazon ec2, to just 2 dedicated servers < $1k/month in an unlimited 1gig colo.
the performance difference i see is huge. i never realized how actually slow EC2 is. their issue is definitely in slow IO, which i think only their SSD instances can fix (which will cost me in the $8k+).
i now have a wicked setup, Xenserver Cloud, SAN, all Highly Available, don't have to worry about bandwidth overcharges, and much much faster .
Particularly now that we have 6 GBps SATA and cheap(ish) 512 GB SSD drives, the I/O differential can be enormous.
I thought about putting four 512 GB SSDs drives in a Raid 10, which would give me striping performance levels without losing the mirror, but that seemed like a bit too much overkill. These servers have 4 front drive bays, only 2 are filled with the SSD mirror, so we could decide to drop in 2 more drives and rebuild the array if we need even more I/O perf.
i now have a wicked setup, Xenserver Cloud, SAN, all Highly Available, don't have to worry about bandwidth overcharges, and much much faster .