The paper about Bigtable, a new kind of distributed database and one of the most interesting Google innovations (next to Google File System and MapReduce), is available:
"Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving). Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the simple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable."
BigTable is used by many Google services, including Google Analytics, Google Finance, Orkut, Personalized Search, Writely, and Google Earth.
September 1, 2006
Subscribe to:
Post Comments (Atom)
its sound very interesting !!!
ReplyDeletedo you know whats the DB engine in the botton ?
what kind of technology is used ?
(I guess that you have very good info...)
It resembles a database, but it's not quite a database. It doesn't support the full relational DB model. And it doesn't speak SQL; but Sawzall, a scripting language used for parallel data processing. The data elements are simply strings of characters, so each app has to know the structure of the data.
ReplyDeleteBigtable uses the Google File System to store data.
More to read:
Google File System
Sawzall
that's fairly cool stuff, i studied similar more primitive systems that were available when in college...nice to see distributed computing done well!
ReplyDeleteAnyone looking for the BigTable white paper can now find it here : http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en//archive/bigtable-osdi06.pdf
ReplyDelete