Let's put four objects in riak:

hb/first will link to hb/second and hb/third hb/second will link to hb/fourth hb/third will also link to hb/fourth hb/fouth doesn't link anywhere

$ curl -X PUT -H "content-type:text/plain" \
  -H "Link: </raw/hb/second>; riaktag=\"foo\", </raw/hb/third>; riaktag=\"bar\"" \
  http://localhost:8098/raw/hb/first --data "hello"

$ curl -X PUT -H "content-type: text/plain" \
  -H "Link:</raw/hb/fourth>; riaktag=\"foo\"" \
  http://localhost:8098/raw/hb/second --data "the second"

$ curl -X PUT -H "content-type: text/plain" \
  -H "Link:</raw/hb/fourth>; riaktag=\"foo\"" \
  http://localhost:8098/raw/hb/third --data "the third"

$ curl -X PUT -H "content-type: text/plain" \
  http://localhost:8098/raw/hb/fourth --data "the fourth"

Now, say we wanted to start at hb/first, and follow all of its outbound links. The easiest way to do this is with the link-walker URL syntax:

$ curl http://localhost:8098/raw/hb/first/_,_,_

The response will be a multipart/mixed body with two parts: the hb/second object in one, and the hb/third object in the other:

--N2gzGP3AY8wpwdQY0jio62L9nJm
Content-Type: multipart/mixed; boundary=3ai6VRl4aLli3dKw8tG9unUeznT

--3ai6VRl4aLli3dKw8tG9unUeznT
X-Riak-Vclock: a85hYGBgzGDKBVIsTKLLozOYEhnzWBn+H/h5hC8LAA==
Location: /raw/hb/third
Content-Type: text/plain
Link: </raw/hb>; rel="up", </raw/hb/fourth>; riaktag="foo"
Etag: 5Fs0VskZWx7Y25tf1oQsvS
Last-Modified: Wed, 24 Feb 2010 15:25:51 GMT

the third
--3ai6VRl4aLli3dKw8tG9unUeznT
X-Riak-Vclock: a85hYGBgzGDKBVIsLEHbN2YwJTLmsTLMPvDzCF8WAA==
Location: /raw/hb/second
Content-Type: text/plain
Link: </raw/hb>; rel="up", </raw/hb/fourth>; riaktag="foo"
Etag: 2ZKEJ2gaT57NT7xhLDPCQz
Last-Modified: Wed, 24 Feb 2010 15:24:11 GMT

the second
--3ai6VRl4aLli3dKw8tG9unUeznT--

--N2gzGP3AY8wpwdQY0jio62L9nJm--

It's also possible to express the same query in map-reduce, directly:

$ curl -X POST -H "content-type:application/json" \
  http://localhost:8098/mapred --data @-
{"inputs":[["hb","first"]],"query":[{"link":{}},{"map":{"language":"javascript","source":"function(v)
{ return [v]; }"}}]}
^D

That's the exact same query. The content type of the response is different. It's now a JSON array with two elements: a JSON encoding of the hb/second object, and a JSON encoding of the hb/third object. (Pretty-printed here, for clarity.)

[
    {
        "bucket": "hb",
        "key": "second",
        "vclock": "a85hYGBgzGDKBVIsLEHbN2YwJTLmsTLMPvDzCF8WAA==",
        "values": [
            {
                "metadata": {
                    "Links": [
                        ["hb","fourth","foo"]
                    ],
                    "X-Riak-VTag": "2ZKEJ2gaT57NT7xhLDPCQz",
                    "content-type": "text/plain",
                    "X-Riak-Last-Modified": "Wed, 24 Feb 2010 15:24:11 GMT",
                    "X-Riak-Meta": []
                },
                "data": "the second"
            }
        ]
    },
    {
        "bucket": "hb",
        "key": "third",
        "vclock": "a85hYGBgzGDKBVIsTKLLozOYEhnzWBn+H/h5hC8LAA==",
        "values": [
            {
                "metadata": {
                    "Links": [
                        ["hb","fourth","foo"]
                    ],
                    "X-Riak-VTag": "5Fs0VskZWx7Y25tf1oQsvS",
                    "content-type": "text/plain",
                    "X-Riak-Last-Modified": "Wed, 24 Feb 2010 15:25:51 GMT",
                    "X-Riak-Meta": []
                },
                "data": "the third"
            }
        ]
    }
]

Another interesting query is “follow only links that are tagged foo.” For that, just add a tag field to the link phase spec:

$ curl -X POST -H "content-type:application/json" \
  http://localhost:8098/mapred --data @-
{"inputs":[["hb","first"]],"query":[{"link":{"tag":"foo"}},{"map":{"language":"javascript","source":"function(v)
{ return [v]; }"}}]}
^D

Here you should get a JSON array with one element: a JSON encoding of the hb/second object. The link to the hb/third object was tagged bar, so that link was not followed. The equivalent URL syntax is:

$ curl http://localhost:8098/raw/hb/first/_,foo,_

It's also possible to filter links by bucket by adding a bucket field to the link phase spec, or by replacing the first underscore with a bucket name in the URL format. But, all of our example links point to the same bucket, so hb is the only interesting setting here.

Link phases may also be chained together (or put after other phases if those phases produce bucket/key lists). For example, we could follow the links all the way from hb/first to hb/fourth with:

$ curl -X POST -H "content-type:application/json" \
  http://localhost:8098/mapred --data @-
{"inputs":[["hb","first"]],"query":[{"link":{}},{"link":{}},{"map":{"language":"javascript","source":"function(v)
{ return [v]; }"}}]}
^D

(Notice the added link phase.) If you run that, you'll find that you get two copies of the hb/fourth object in the response. This is because we didn't bother uniquifying the results of the link extraction, and both hb/second and hb/third link to hb/fourth. A reduce phase is fairly easy to add:

$ curl -X POST -H "content-type:application/json" \
  http://localhost:8098/mapred --data @-
{"inputs":[["hb","first"]],"query":[{"link":{}},{"link":{}},{"reduce":{"language":"erlang","module":"riak_mapreduce","function":"reduce_set_union"}},{"map":{"language":"javascript","source":"function(v)
{ return [v]; }"}}]}
^D

The resource handling the URL link-walking format does just this:

$ curl http://localhost:8098/raw/hb/first/_,_,_/_,_,_

That should get you just one copy of the hb/fourth object.

So why choose either map/reduce or URL-syntax? The advantage of URL syntax is that if you're starting from just one object, and just want to get the objects at the ends of the links, and you can handle multipart/mixed encoding, then URL syntax is much simpler and more compact. Map/reduce with link phases should be your choice if you want to start from multiple objects at once, or you want to get some processed or aggregated form of the objects, or you want the result to be JSON-encoded.

Riak version 0.8 note: In Riak 0.8, the format of the result of 'link' map/reduce phases was not able to be transformed into JSON. This meant both that it was not possible to put a Javascript reduce phase right after a link phase, and also that it was not possible to end an HTTP map/reduce query with a link phase. Those issues have been resolved in the tip of the source repository, and will be part of the 0.9 release.

Riak version 0.9 note: In Riak 0.9, the URL for Riak's HTTP interface will change from /raw to /riak. You may need to adjust the above examples accordingly.

-Bryan

(Note: I assume that you have successfully built an instance of Riak for your platform. If you built Riak from source in ~/riak, then set $RIAK to ~/riak/rel/riak.”)

We first get started by grabbing a stable release of Innostore. You’ll need to download the source for a release from: http://bitbucket.org/basho/innostore/downloads/

Looking in the “Tags & snapshots” section, you should download the source for the highest available RELEASE_* tag. In my case, RELEASE_4 is the most recent release, so I’ll grab the bz2 file associated with it:

http://bitbucket.org/basho/innostore/get/RELEASE_4.tar.bz2


Once I have the source code, it’s time to unpack it and build:

$ tar -xjf innostore-RELEASE_4.tar.bz2
$ cd innostore
$ make


Depending on the speed of the machine you are building on, this may take a few minutes to complete. At the end, you should see a series of unit tests run, with the output ending:

=======================================================
All 7 tests passed.
100222 7:43:58 InnoDB: Shutdown completed; log sequence number 90283
Cover analysis: /Users/dizzyd/src/public/innostore/.eunit/index.html


Now that we have successfully built Innostore, it’s time to install it into the Riak distribution:

$ ./rebar install target=$RIAK/lib

If you look in the $RIAK/lib directory now, you should see the innostore-4 directory alongside a bunch of .ez files and other directories which compose the Riak release.

Now, we need to tell Riak to use the Innostore driver as a backend. Make sure Riak is not running. Edit $RIAK/etc/app.config, setting the value for “storage_backend” as follows:

{storage_backend, innostore_riak},

In addition, append the configuration for the Innostore application after the SASL section:

{sasl, [ ....
]}, %% < -- make sure you add a comma here!!

{innostore, [
{data_home_dir, "data/innodb"}, %% Where data files go
{log_group_home_dir, "data/innodb"}, %% Where log files go
{buffer_pool_size, 2147483648} %% 2G in-memory buffer in bytes
]}

You may need to adjust the directories for your data_home_dir and log_group_home_dirs to match where you want the inno data and log files to be stored. If possible, make sure that the data and log dirs are on separate disks -- this can yield much better performance.

Once you've completed the changes to $RIAK/etc/app.config, you're ready to start Riak:

$ $RIAK/bin/riak console

As it starts up, you should see messages from Inno that end with something like:

100220 16:36:58 InnoDB: highest supported file format is Barracuda.
100220 16:36:58 Embedded InnoDB 1.0.3.5325 started; log sequence number 45764

That’s it! You’re ready to start using Riak for storing truly massive amounts of data.

Enjoy,

Dave Smith

Enter Sean Cribbs. As Sean details in his latest blog post, "You Got your Riak in my Ruby," Basho and the stellar team at Sonian made it possible for him to hack on Ripple, a one-of-a-kind client library and object mapper for Riak. The full feature set for Ripple can be found on Sean's blog, but highlights include a DataMapper-like API, an easy builder-style interface to Map/Reduce, and near-term integration with Rails 3.

And, in case you need any convincing that you should consider Riak as the primary datastore for your next Rails app, check out Sean's earlier post, "Why Riak should power your next Rails app."

So, if you've read enough and want to get your hands on Ripple, go check it out on GitHub.

If you don't have Riak downloaded and built yet, get on it.

Lastly, you are going to be seeing a lot more Riak in your Ruby. So stay tuned because we have some big plans.

Best,

To answer that question and fill that need, Innostore was written. It is a standalone Erlang application that provides a simple interface to Embedded InnoDB. So far its performance has been quite good, though InnoDB (with or without the Innostore API) is highly dependent on tuning the local configuration to match the local hardware. Luckily, Dizzy -- the author of Innostore -- has some heavy-duty experience doing that kind of tuning and as a result we've been able to help people meet their performance goals using Innostore.

-Justin

Once you are done with the podcast, go download erlang_js.

Enjoy,

Mark Phillips

Right click here to download the Podcast

Also, if you haven't had a chance to join the Rebar mailing list, you can do so here.

Mark

Go forth and consume!

Mark

• Justin Sheehy's Riak presentation at NoSQL East

  Link: https://nosqleast.com/2009/#speaker/sheehy

• Bryan Fink's fantastic overview of Riak in October at SQLFreeNYC

  Link: http://www.vimeo.com/6973519

• Dave Smith's introducion to Riak in an embedded node using Rebar, a packing and build tool for Erlang applications.

  Link: http://vimeo.com/8311407

• Dave Smith also gave a more general overview of Rebar

  Link: http://vimeo.com/8311407

• Martin Scholl's...awesome presentation on Riak at NoSQL Berlin

  Link: http://vimeo.com/7318171

• Rusty Klophaus' presentation entitled "Nitrogen and Riak by Example" at Erlang Factory in Stockholm

  Link: http://bit.ly/5AgKzW

• Bryan Fink's screencast with Ben Ahlan of Video Code Chat demonstrating basic setup and usage of Riak

  Link: http://bit.ly/akagc