Erlang Everywhere

Brief History of CouchDB

  • Document Oriented Database
  • Started by Damien Katz: 2005
  • Originally XML Database written in C++
  • Rewritten in Erlang, moved to JSON
  • Became an Apache project 2008
  • Founded Couch.io 2010 -> CouchOne -> Couchbase

What is CouchDB?

  • JSON
  • HTTP
  • Map Reduce
  • Replication (Sync)
  • CouchApps
  • GeoCouch
  • Robust

JSON:

{
    "_id": "49c8b92961f5fbf9bc13fbcc00000484",
    "_rev": "2-bc797d9090695d916a89d6166d125581",
    "some": "json",
    "an": ["array", "of", "things"],
    "and": {"child": "objects"},
    "numbers": 42
}

HTTP API - Database

GET $HOST
{"couchdb":"Welcome","version":"1.2.0a-c856916-git"}

GET $HOST/demo
{"error":"not_found","reason":"no_db_file"}

PUT $HOST/demo
{"ok":true}

GET $HOST/demo
{"db_name":"demo","doc_count":0 ... }

HTTP API - Documents

PUT $HOST/demo/doc {"foo":"bar"}
{"ok":true,"id":"doc","rev":"1-4c6114c65e295552ab1019e2b046b10e"}

GET $HOST/demo/doc
{"_id":"doc","_rev":"4-42dd533bd9d3e47403c1914ad663ab8b","foo":"bar"}

POST $HOST/demo '{"foo":"bar"}'
{"ok":true,"id":"49c8b92961f5fbf9bc13fbcc0000128a", \
  "rev":"1-4c6114c65e295552ab1019e2b046b10e"}

HTTP API - Documents

PUT $HOST/demo/doc
  {"_rev":"1-4c6114c65e295552ab1019e2b046b10e", "new":"content"}
{"ok":true,"id":"doc","rev":"2-c8b28721ed96f3dc039a1ab3f85b0dba"}

DELETE $HOST/demo/doc?rev=2-c8b28721ed96f3dc039a1ab3f85b0dba
{"ok":true,"id":"doc","rev":"3-8d9fecfc76473aa0d412ca228468c8cc"}

PUT $HOST/demo/newdoc/test.html --data-binary @test.html
{"ok":true,"id":"newdoc","rev":"1-a2833f8c791e5ce692f2210ef3249d89"}

Map Reduce

Map Reduce

Incrementally Built
Map
function(doc) {
  if (doc.type && doc.type === 'task' && doc.status === "complete") {
      emit([doc.index || 0], null);
  }
}
Reduce
_count

BTree

  • Uses B-Tree for both data and view storage
  • Minimise disk seeks / Concurrent access

Append Only

  • BTree is stored on disk as append only file
  • Updates / Deletions dont modify data
  • Only returns ok for writes when data is guaranteed to be
    safe on disk (delayed_commits)
  • killdashnine party! Safe way to shutdown CouchDB is
    kill -9

Changes Feed

Can read the database as a series of changes
$ curl $HOST/demo/_changes
{"results":[
{"seq":2,"id":"49c8b92961f5fbf9bc13fbcc0000128a","changes":
  [{"rev":"1-4c6114c65e295552ab1019e2b046b10e"}]},
{"seq":7,"id":"newdoc","changes":
  [{"rev":"1-a2833f8c791e5ce692f2210ef3249d89"}]},
{"seq":8,"id":"doc","changes":
  [{"rev":"4-42dd533bd9d3e47403c1914ad663ab8b"}]}
],
"last_seq":8}
Continous Changes can notify database activity
curl "$HOST/demo/_changes?since=8&feed=longpoll"

Replication (Sync)

Move data between databases
POST /_replicate {"source": "mydatabase", "target":"backup"}
Can specify specific documents / generic filter
function (doc, req) {
  if (doc._id.match("_design")) {
    return true;
  }
  return doc.type && doc.type === "task";
}
Can run continuously (changes are updated in real time)

GeoCouch

GET $HOST/places/_design/main/_spatial/points?bbox=0,0,180,90
R-Tree Index
GeoSpatial Queries
MapQuery

CouchApps

Mochiweb web server built in
Can serve files via attachments
Can build whole Web apps with just CouchDB

Couch in the wild

Canonical have built a file sync into Ubuntu itself, based on CouchDB it lets users access their files and program settings wherever they are.


Meebo, one of the most popular online chat applications, uses CouchDB to store its users logs for future access and analysis.

Couch in the wild

Dimagi uses CouchDB to provide healthcare data to remote clinics with no reliable internet.


Radical Dynamic built Group Complete, a mobile app for offline data collection on Android

Why Mobile

  • Phones are often offline, need local data and ability to sync
  • Phone can kill app at any time, needs robust storage
  • HTML5 and Geo capabilities

Sync Patterns

  • Couch has flexible / powerful sync

Sync Patterns - Follow / Privacy

Sync Patterns - Group Share

Android Overview

  • Modified glibc
  • Different shell (/system/bin/sh)
  • Lack of system libraries (blowfish)
  • Lots of fixing build files
  • Cant access beam files via filesystem

iOS Overview

  • No Dynamic linking
  • No Subprocesses (DNS and NIFs)
  • OS can kill the process at any time
  • Cant execute writeable memory

Erlang on Mobile

  • Mobile Networks will try to 'optimise' traffic
  • No way to reduce build reliably
  • Strip debug symbols
  • Surpisingly good power usage

Going Forward

  • Get patches into Erlang + Apache CouchDB
  • Reduce Build Size
  • Test thoroughly
  • Improve Startup Speed
  • General Release Imminent

Thank You!

Questions?