Liveblogging: A dozen databases in 45 minutes

Posted in: Cassandra, Technical Track

Actually, the slide is 12 databases in 25 minutes (and 20 minutes of theory) by Eric Redmond (@inviite).

Complex data:  A lot of data isn’t really complex, it’s just modeled in a complex way.

“Complexity is a symptom of confusion, not a cause” Jeff Hawkins.


Linear Scalability

Ability to be Distributed

Low Latency





ACID (transaction-based)

Atomic – Transactions are “all or nothing”

Consistent – system data will have integrity

Isolated – Transactions can’t see each other

Durability – changes aren’t lost


BASE (request based)

Basically Available

Soft state

Eventual consistency

Redmond says: ACID is negative (avoid negative things), BASE is mostly positive, things will be good, not great.

CAP theorem – it’s a fact :D

Consistent, Available, Partition-tolerant web services.  “It is impossible to reliably provide atomic, consistent data when there are partitions in the network.  It is feasible, however, to achieve

Note that “consistent” is not the same as “consistent” in ACID, it’s more like Atomicity.

Strong consistency: when an update completes, subsequent access returns the new result.  [to my mind this is actually durability]

Weak consistency – eventual consistency

“Correct consistency” – is the most important part.  DNS, for example, is eventual consistency.

Common patterns:


CouchDB has an amazing ability to do this, Mongo is also good but not as good.

– copying data amongst nodes in a distributed database.  Lazy (optimistic) replication, gossip (nodes communicate to stay in sync). – master/slave (mongo)

– master/master (riak, couch)

– vector clocks (keep track of write order per client

– mvcc (mysql)


N – Nodes to write to (per bucket)

R – Nodes read from before success

W – Nodes written to before success

Amazon Dynamo does this (Cassandra and Riak do this) – supports both CP and AP in one db (from the CAP theorem)

Consistent Hashing

Balance your servers, and when you hash your keys, if a server goes down or is added you don’t have to rebalance ALL nodes, just some % of them.



Relational Models:

“Nothing beats relational databases for raw queryability.”  The tradeoff — you have to structure your data and tell the system how it is structured.

PostgreSQL (full featured) –,,

MySQL (lighter) –,

Drizzle (lightest) –


Bigtable/Columnar Style

What makes it columnar?  well, a primary key is really a row key, and then you have column families, which are columns, stored together.  (each column’s values are stored together as opposed to the row being stored together.)  You can set expiry for a column family too, after which the data expires (which is why it’s great for Google).

HBase – – Google’s BigTable implementation, which was born of Hadoop (Java mapreduce engine).  If you want to use HBase in production, use Thrift ( which Cassandra also uses). This is CP, but configurable to AP.  Does sequential reads and column versioning, strong but flexible columnar schema.

Cassandra – hybrid.  Node architecture like dynamo – data structure like BigTable w/column families – – Good for hundreds of nodes in the same data center, if there is more than that or different data centers, use HBase (that’s what Digg and Facebook are running into).  In cassandra you set up your schemas with an XML file, not with DDL.  Benefits – sequential reads of ordered keys, also has versioning.  It’s AP, configurable to CP.


Documentation Datastores:

MongoDB (AP focused – master/slave) – created to be huge (huMONGous).  Made to be partitioned, distributed, needed ad hoc queries.  Wasn’t built to be durable.



Not made to be distributed, originally, was meant to be very durable.  AP focused (master/master) (clustering)

MapReduce in Mongo is an ad hoc query, comfortable for relational db ppl.  In CouchDB, you make views and then request data from those views.

Riak – The most “architecturally cool” database out there.  It’s a dynamo implementation that is purely REST based.  It’s a key-value store, but it’s not descriptive enough — it has map-reduce built in, metadata and links you can walk.  You can store ANYTHING in riak — not just text.  example: getting a JPG file from the web and putting it as the value for the key “firefox.jpg”.  Neat demo.

Riak has a ring, eventual consistency, can pull nodes in and take nodes out, without having to invalidate all the ids.  It has quorum consistency, which blows Eric’s mind, but we didn’t have


Key/value stores

memcached – don’t use it

Kyoto Cabinet – don’t use it

Redis – use it – – it can handle lists, hashes, can intersect the value of 2 keys (such as person and pet, to find out who owns which set).


Graph datastores – you walk the graph instead of querying or doing mapreduce.


FlockDB  – distributed, “unless you’re twitter, you don’t need to use it”.  It’s not really possible to distribute a graph database, you can’t walk it and do node traversals, you can just walk edges (you can do friends, but not friends of friends, etc).

Slides are available at


Interested in working with Sheeri? Schedule a tech call.

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *