Dropbox declares “We don’t need no Amazon Web Services!” Well, not really, but you get the idea.
I’ll bet most of you have used Dropbox. Back in 2007, Drew Houston, Dropbox’s founder and current CEO, had gotten sick and tired of misplacing his USB drive. So, he hacked together the first, easy-to-use personal cloud storage. Today, even though Dropbox offers less free storage than other personal cloud drives, with half-a-billion users Dropbox is still very popular.
That part you probably already knew. What you probably didn’t know was that your Dropbox files weren’t in a Dropbox data center. Instead, they were hiding out on Amazon Simple Storage Service (Amazon S3).
As Akhil Gupta, Dropbox’s VP of infrastructure engineering, explained, “We were an early adopter of Amazon S3, which provided us with the ability to scale our operations rapidly and reliably. Amazon Web Services has, and continues to be, an invaluable partner – we couldn’t have grown as fast as we did without a service like AWS.
“As the needs of our users and customers kept growing, we decided to invest seriously in building our own in-house storage system. There were a couple reasons behind this decision. First, one of our key product differentiators is performance. Bringing storage in-house allows us to customize the entire stack end-to-end and improve performance for our particular use case. Second, as one of the world’s leading providers of cloud services, our use case for block storage is unique. We can leverage our scale and particular use case to customize both the hardware and software and provide better unit economics.”
Technically, Dropbox has always had a hybrid cloud architecture. It hosted its metadata and web servers on its own data centers, while storing file content on Amazon S3. Now, to improve customer performance, Dropbox is bringing storage in-house to its own private cloud: Magic Pocket.
Magic Pocket is meant to ramp up to being an exabyte-scale storage system. Dropbox began it as a proof-of-concept in 2012. After testing it, the company deployed it as a “dark launch” in 2014. Then, on February 27, 2015, Dropbox began storing and serving user files exclusively in-house. Over the next six months, Dropbox ramped up the system to more than 500 PetaBytes.
By October 30, 2015, ahead of schedule, Dropbox was hosting 90% of U.S. data from its in-house servers.
That isn’t to say Dropbox has abandoned S3. It hasn’t.
With 75% of its users outside the U.S., and many of those in Europe, Dropbox will start hosting European customer data by the third quarter of 2016. Dropbox will be doing this in an AWS data-center in Germany.
The real story isn’t how Dropbox is using the technology. It’s whether using a hybrid cloud will enable the company to reach a point where its incremental costs go down relative to a public cloud provider. In short, can a hybrid cloud approach be profitable for a company offering “public” cloud services?
The answer to that question is going to be one that both Dropbox and many other companies will be interested in knowing.