I keep reading about people saying AWS is cheap. Do these people actually use AWS? Even with price drops, it will never be as cheap as buying a machine and sticking it in a co-lo and self managing it. The problem is that's a pain in the ass.
The real reason why people use AWS is because it's convenient.
I think the price cutting / value proposition is beginning to finally outrun the needs that the extreme majority of sites might have. For most of the US Web, infrastructure demand is not growing, and neither is traffic (the principle that there are only so many hours to consume content, against a mostly maxed out user base). Your typical business web site, will never keep up with what Amazon is doing. So at some point in the next few years, Amazon will be both cheap enough and convenient, versus co-lo or dedicated.
In the last year I've noticed AWS is finally catching my eye on price, and I'm a big co-lo self-managed fan due to the cost savings. In a few more years I could see myself switching everything over and not feeling bad about the cost.
In that time Amazon's value proposition will chase down toward where Linode and Digital Ocean are at today. Let's say their system power (the sum of the hardware) will go up by 50% every year, while their prices fall 25% every year per 2014 defined unit of power. Even though I consider Amazon expensive today, it won't be long at this point before they start to outstrip my scale of service demand (I'll never be operating Twitter or Foursquare on AWS).
The US market may not be growing, but the rest of the world will come online soon enough. And speaking from experience, most of the world lacks the infrastructure, climate, political stability, or protection from criminal activity to host individual server farms.
If you're trying to start a business in India or Egypt, you're VASTLY better off outsourcing your IT to a dependable third party like AWS -- particularly as those costs drop over the next decade through economies of scale and technological progress. I'm sure the eventual goal is to sell AWS and its competitors as replacements for virtually all business IT and even personal computers -- even in stable Western countries. Current prices may have finally started to be competitive with local hosting, but we haven't even touched the potential of cloud computing yet.
> as cheap as buying a machine and sticking it in a co-lo and self managing it
For most companies out there, that involves hiring someone with Linux/Windows server management skills. Even in an engineering company, if you have a choice between spending engineer's time on engineering projects or system management, the company usually gets more bang for the buck for the former.
From personal experience, I've seen a correlation between software companies that "outsource" their devops and software companies that write bad software. This is more pronounced the more "magic" you get (Heroku for example).
Put differently, the best backend developers are also the ones that can compile nginx from source (a rare talent nowadays apparently) and have it running reliably within the hour. Better by no small factor either. There's no shortage of them.
Yeah, the word "companies" describes everything from a one-man shop to Google, and at certain point a devops person/department/division is a must. Platform-as-a-service companies are trying to prolong that "certain point".
A few years back you could have the same argument with an army of Exchange administrators who would point out how ridiculous it would be to trust your entire company's email infrastructure to Gmail, yet here we are.
Isn't compiling nginx from source just a `./configure; make; make install`? I mean, any developer who has ever used a *nix-like probably knows the classical three-command incantation.
Last time I was part of the process to interview devops, we asked this question (pairing remotely tmux'd into an ec2 instance). I think we went through 10 people before we found 1 that could do it. And, of course, when he did it, it took him like 5 minutes. Everyone else struggled. "but I use a package manager" was the most common complaint.
You do need things like build-essential installed (which we'd let them install however they wanted). And a blank ./configure normally complains about missing pcre and maybe gzip (I forget). But we'd just tell them they could install without it (and the error message makes it very clear how to do that (--without-pcre or whatever).
Very true, but if you're using EC2 you're still going to need those skills anyways. It's not like those instances manage themselves. If you've ever seriously used EC2 to do anything, you've probably already got those skills anyway.
What IaaS really saves you is the tedium of dealing with the physical aspects of servers. No requisitioning hardware, driving to co-los, dealing with phasing hardware out, etc.
> it will never be as cheap as buying a machine and sticking it in a co-lo and self managing it.
if your definition of cheaper is having to cough up the large upfront capital to rent/buy a place, rent/buy a rack(s), and then pay somebody (or DIY) to install, hook up power and test disaster recovery options etc...then yes, it's cheaper.
If you have a 5 or 10 person start-up, it's drastically cheaper to find a high quality dedicated host that can make sure all of that is operating for you. Unless you have very specialized needs of course.
The amount of dedicated hardware you can rent for $1,000 per month is several times beyond what you can get from Amazon for the same price. And if you have a demand for lots of bandwidth, Amazon will cost more just on bandwidth than for the entire dedicated setup somewhere else.
Amazon will drill you for $1,000+ for just 10tb of bandwidth alone. Today 10tb of bandwidth on a 1gbps line is almost the assumed base you get with a $100-$150 dedicated box.
I agree that storage, which is to be the focus on this article, seems to be a race to the bottom. But when it comes to computing, EC2 and friends are still many times less efficient (price/perf) than non-cloud solutions (cheap vps', dedicated or colo). A lot of these value added services sit on top of this infrastructure, so you end up paying quite a bit more than you would have in a non-cloud era.
But people say: well, you're outsourcing your devops. This is absolutely false when it comes to the pure compute offering. At best, an EC2 instance is as easy to manage as anything else. It's more true of things like SQS, SES, RDS, DynamoDB, ... though personally, I cringe when a team of developers can't manage their own production postgresql or rabbitmq servers.
Still, clearly people are seeing some value in these services. Softlayer, Rackspace and friends dropped the ball here. 15 years of fat profits rendered them sterile. Go back 6 years, sell the performance of bare metal, add quality and affordable managed services (not a $1000/m load balancers), and you might have been able to compete.
Case in point: Softlayer's listed prices are still complete bullshit. If you work their sales, you can get servers for 50-60% less even on small orders. But you need to spend 10 days emailing back and forth. Can you even get prices from Rackspace's website? I just went on their site and it made dell.com look useful.
at best, an EC2 instance is as easy to manage as anything else
If you use EC2 in a very basic way, yes. When you start to factor in things like CloudFormation, auto-scaling, elastic load balancing, route 53 DNS, IAM roles, there is a lot of just plain cool shit as part of the AWS ecosystem.
My point was that there was nothing stopping every other company from doing this. Not in 2006, certainly not in 2012.
Some of those companies, then and now (??), are better positioned to truly drive towards zero because of how awful the price/perf is of cloud hosting. Storage aside, it's DO/Linode/OVH/Hetzner, not Amazon, that's driving towards zero.
On an aside:
Route 53's innovation is that, in huge number of cases, it's dirt cheap. But there have been relatively good and affordable DNS solutions before and since. I am a Route53 fan though, in large part because it's decoupled from AWS. You can use it (and S3) while avoiding everything else from AWS.
Auto-scaling is what a sales engineer tells a C[ITE]O in exchange for a signed contract. Scaling comes from design/"architecture"...you can't auto scale your way out of bad design: stateful services, SELECT N+1, n^2 algorithms, LIKE '%spice', poor/unknown cache hit ratios and so on.
I personally think that AWS auto-scale groups are poorly named, as they can do more than just the sort of brute force "scaling" that you are rightfully disdainful of.
You can, for example, create an auto scale group to ensure a minimum of n healthy instances of an application, so if one falls over, it will be replaced with a pristine one.
To me, auto-healing is a lot more interesting than auto-scaling.
I expected a bit more insight from such an ominous title. Long story short: cloud storage and computation are a commodity; cloud companies need to offer value-added features to turn a profit.
Good summary, but it is happening. Cloud storage is a commodity, and now almost free.
The problem, not addressed in this article, is that it is not free. Microsoft hope to recoup the costs of giving storage away through Office purchases. Google perhaps through knowing more about Google Drive users and selling targeted ads.
The medium-sized companies like Dropbox and Box may be the ones squeezed out.
No doubt he has. Box is currently suffering through the implications of it all - they're bleeding cash at a business-threatening rate. If this era of historically cheap money started to fade tomorrow, Box would go bankrupt very quickly.
I highly doubt that's in the cards for Box. Their entire business model is around selling security and enterprise features.
In that sense, I do agree that the future is probably that storage is totally free (at least for consumerized apps—S3 will never be free). Companies can make their profits on value-added services like security (Box) or editing tools (Microsoft).
So at some point storage costs will hit a pretty inelastic floor price and then things will get very interesting for companies that advertise free or very low cost "unlimited" storage as their monthly costs will increase with every extra GB that their customers upload.
It's safe to predict that other photo storage & sharing sites will follow Everpix in going under. E.g., Shoebox (http://shoeboxapp.com/) seems to have the exact same business model as Everpix so I can't see them surviving long-term unless they kill their free plans.
David Rosenthal (http://blog.dshr.org/) also has lots of good info on the economics of long-term storage.
Let's hope they keep racing to the bottom. To break down the value of each brand to consumers:
Dropbox
$120/year for 1 account, 1TB
No photo improvement or sharing features
No document interface (aside from coming compatibility with Office)
Google Drive
$120/year for 1 account, 1TB
G+ photo sharing and auto-awesome improvements
Good-enough GDocs and best-in-class real-time editing
OneDrive
$100/year for 5 accounts, "unlimited" storage (20k file limit)
No photo improvement or sharing features
Best-in-class Office and good-enough real-time editing
Amazon Cloud Drive
$100/year for 1 account, unlimited photo storage
No photo improvement or sharing features, but good timeline organization (and deduplication?)
No document interface
Bundled with Prime shipping/videos/music/lending library
- - - - -
Of the above, Dropbox offers no real value aside from great functionality. That it "just works" SHOULD be a given, but given that OneDrive is capped at 20,000 files and fails to upload from my desktop, I guess we shouldn't take that for granted.
GDrive works fantastically well and offers great photo enhancement, but the price is the highest of anyone here. Particularly if they're trying to develop G+ into a Facebook competitor, you'd think they would offer more-than-competitive pricing to encourage consumers to keep their photo libraries online and glean information about consumer movement patterns, personal networks, local attractions, and personal brand interests. Why roll out cool auto-awesome features if you're going to price them out of the market?
OneDrive has the second best value behind Amazon Prime... in theory. In practice, OneDrive has a 20,000 file cap and again, the program simply won't upload my files from my desktop. There's no photo enhancement, sharing, or organization. I guess they're relying on Office as the big draw. Which makes sense -- documents are an insignificant burden, allowing for steady revenue with few attached costs. And most people won't understand that their photo libraries weren't safely uploaded to the cloud as they expected.
Amazon's Prime Photos is a GREAT value (and seems to work really well in uploading and organizing my photos), but I'm still unclear on whether Prime deduplicates my uploads or has excellent sharing/privacy controls like you can find on G+. Still, the photos feature makes Prime a must-buy even if the rest of Prime's insane value didn't already do that. The only catch is that Amazon doesn't offer two-factor authentication... which is a big deal, Amazon!
- - - - -
Given that the company with the biggest consumer base can leverage their audience to build out the biggest, baddest infrastructure that subsequently takes advantage of marginal cost economics in the delivery of their own services or in the replacement of corporate IT... the race to the bottom makes perfect sense in the consumer sphere. You want the biggest consumer base to justify short-term expansion (and then finance that expansion in the long term with your locked-in customers).
This is only a race to the bottom if you assume that consumer storage is the end game of clouds. In ten years, the biggest clouds can theoretically provide services that replace most corporate IT... and even most personal computers, as seen in today's thin-client Chromebooks. And if Microsoft lets Google beat Azure in this arena, not only will Google dominate in the field of cloud services, Google's vast infrastructure could leverage economies of scale to deliver Google's search/local/maps/whatever services vastly quicker and cheaper than Microsoft could.
Clouds are the modern-day railroads. It's not a race to the bottom if you're investing to replace individual ox-drawn wagons with your own infrastructure.
I believe google drive also has partially unlimited photo storage, if the photos are below a certain resolution they don't count towards your storage quota.
That's true -- photos under 2000px are free, and Google will automatically resize full-size photos as they're uploaded. Personally, I just upload my entire photo library to GDrive and G+ automatically processes them into G+ Photos. Hopefully Google rolls out unlimited full-size photo storage soon, since even I'm not sure why I pay for GDrive when I already have Amazon Prime.
In an ideal world, G+ Photos will roll out with unlimited full-size storage that deletes both smaller-resolution duplicates already in G+ Photos and also deletes full-size duplicates taken from my GDrive. I don't even mind the vendor lock-in of G+, where it's fantastically difficult to download or delete your entire library. I just want a single, organized-by-date, comprehensive location for my entire library that allows me to hoard, peruse, improve, and share my photos. G+ is so damn close to that dream already...
I wrote off Flickr a long time ago and didn't even consider them for this comparison. My mistake.
Does Yahoo run Flickr on its own infrastructure, or rent it like Dropbox and Apple? If the former... why isn't Yahoo a major player in this market? Giving away a terabyte is great, but you would think that it signals a move by Yahoo to build out their infrastructure.
Please don't neglect to notice that this is not actually an article so much as a list of companies with cloud storage products, and that it doesn't even bother discussing the critical part of the title (that the race to zero is scaring companies).
Meanwhile, with AWS services are still expensive, especially bandwidth. While storage providers may be willing to subsidize my stuff, my data providers are my real cap in usage.
The real reason why people use AWS is because it's convenient.