An unofficial blog that watches Google's attempts to move your operating system online since 2005. Not affiliated with Google.

Send your tips to gostips@gmail.com.

September 1, 2006

The Paper About Bigtable

The paper about Bigtable, a new kind of distributed database and one of the most interesting Google innovations (next to Google File System and MapReduce), is available:

"Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving). Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the simple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable."

BigTable is used by many Google services, including Google Analytics, Google Finance, Orkut, Personalized Search, Writely, and Google Earth.

4 comments:

  1. its sound very interesting !!!

    do you know whats the DB engine in the botton ?
    what kind of technology is used ?

    (I guess that you have very good info...)

    ReplyDelete
  2. It resembles a database, but it's not quite a database. It doesn't support the full relational DB model. And it doesn't speak SQL; but Sawzall, a scripting language used for parallel data processing. The data elements are simply strings of characters, so each app has to know the structure of the data.

    Bigtable uses the Google File System to store data.

    More to read:
    Google File System
    Sawzall

    ReplyDelete
  3. that's fairly cool stuff, i studied similar more primitive systems that were available when in college...nice to see distributed computing done well!

    ReplyDelete
  4. Anyone looking for the BigTable white paper can now find it here : http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en//archive/bigtable-osdi06.pdf

    ReplyDelete

Note: Only a member of this blog may post a comment.