Google BigTable: A Distributed Structured Storage System

Two weeks ago at OSCON Google announced their new offering...open source code hosting. Their initial feautres, although interesting, are not yet a compelling reason to move your project from SourceForge and the like. As all "beta" software, features will expand, be refined, and improve. There were two interesting pieces that I picked on during the presentation that were orthogonal to the annoucement. First is how the developers handled structured data in a "label", semi-structured manner. I'll discuss that in another post as it deservers more attention. The other hidden gem in the annoucement, in my opinion, is BigTable. Announced earlier, BigTable is a distributed structured storage system which Google uses throughout its offerings. For instance, although I have no proof (help me out if you find documentation), I'm sure it is used for Google's Calendar, Mail, Maps, etc. I found the following video that presents the background and basic architecture of BigTable:

In this lecture at the University of Washington, Google's Jeff Dean discusses the Bigtable content storage system used in google's backend.

BigTable: A Distributed Structured Storage System - Google Video