I got the directive this weekend to transfer stuff off of the cloud immediately. We lost a weekend of work for a lot of people, and nothing I can say as a tech will account for that. Upper management wants us the hell off.
I'd argue that the overall cost is much less than having all of these services in house, and in house services go down too.
But I think for the moment, they want someone to yell at, and Amazon gives the most unhelpful lack of communication, with no even remote eta, and that's unacceptable.
I think you'd be surprised at how much better/cheaper you could have done it in house.
Given proper motivation (say 20c bonus on every dollar saved) I think we'd see this argument vanish pretty quickly.
If you listen to the wrong people (sales-guys from "Enterprise Grade" vendors), or pinch too many pennies it can easily be a disaster. It's dangerous water to tread on your own for sure.
Beware of wrong incentives. This is exactly how people are tempted to improve the average case with a trade-off with the worst case. Dollars saved in the short term can cause catastrophic problems later.
Business guys just don't understand any of this stuff. They hear the cloud buzzwords and want to jump on the bandwagon so they can be one of the cool kids, and then they panic when a failure, native and incident to any third-party service, occurs. Really silly.
If it's critical not to have problems if/when Amazon goes down, you have to plan for that ahead of time, just the same as anything else. It's not like throwing the "cloud" label on something makes it invincible.
As computers become increasingly integral to the daily operations of a business, the business guys are not going to have any choice but to learn some basics. We're really already at this point, but maybe after enough failures similar to those witnessed this weekend it will finally sink in.
I am amazed how many people want to blindly follow the buzzword bandwagon without obtaining even a vague notion of the technical implications first. The fact that Amazon would be controlling Amazon EC2 and that if a failure occurs at Amazon, their EC2 product may be affected, is the most blatant thing about using "Amazon EC2" or any other external service provider.
I don't feel like you're adding a lot to the discussion by stereotyping and calling out "business guys". As a comparitively non-technical person to the average hacker news user, I would expect Amazon to quickly and clearly communicate to me why they are unable to provide the contracted service, what they are doing to remedy the issue, when it will be returned, and what we can do to avoid a similar outage in the future. From there, I can determine my risk aversion and the oppurtunity cost of choosing such a solution.
From what I gather, this is not how it was handled (I may be wrong), and for that I could not put my trust in them.
Technical failures in complex applications are not always able to be "clearly communicated". I don't use EC2 in any major capacity so I may be wrong here, but my understanding of the failure is that Amazon acknowledged the system had failed and that they were working to get the systems back up. I don't know what else you expect in the midst of an outage -- the fact is that if the technical failures were foreseen and planned, there wouldn't be an outage, so you have to give the technicians time to figure out what happened and figure out a way to fix it.
The details you want don't come until the crisis is over and the users are back online. Occasionally you may be able to get a meaningful ETA, but it really depends on the nature of the failure(s) that caused the outage. I'm glad Amazon didn't cave to the pressure and just throw a random guess out.
I'd argue that the overall cost is much less than having all of these services in house, and in house services go down too.
But I think for the moment, they want someone to yell at, and Amazon gives the most unhelpful lack of communication, with no even remote eta, and that's unacceptable.