Monday, April 15, 2013

Exploring Riak with F#

I have embraced the Polyglot Programming for quite a while already. This year, I wanted to tackle Polyglot Persistence. RDBMS has dominated my world view of persistence layer with everything else as second class citizens. I thought it was time to expand my world view of persistence layers, especially with the burgeoning popularity of NoSQL movement. To begin my exploration, I picked up the book Seven Databases in Seven Weeks by Eric Redmond and Jim Wilson and started with the first NoSQL persistence layer in the book, which was Riak. According to the book:

Riak is a distributed key-value database where values can be anything-from plain text, JSON, or XML to images or video clips-all accessible through a simple HTTP interface.

Setting up Riak

I deployed riak on 3 servers for testing purposes. In setting up the Riak clusters, I ran into the following errors:

10:42:57.339 [error] gen_server riak_core_capability terminated with reason: no function clause matching orddict:fetch('riak@192.168.56.1', [
{'riak@127.0.0.1',[{{riak_core,staged_joins},[true,false]},{{riak_core,vnode_routing},[proxy,legacy]},...]}]) line 72
/users/domains/riak/lib/os_mon-2.2.9/priv/bin/memsup: Erlang has closed.

A quick Google search brought up the following link Googling the web, I got this link: http://blog.alwayshere.info/2012/11/riak-error-genserver-riakcorecapability.html I originally started riak with 127.0.0.1 address. Then I made the modification to the configuration as documented in http://docs.basho.com/riak/latest/cookbooks/Basic-Cluster-Setup/ in trying to setup my 3 server Riak into a cluster. To fix my error, I had to go to ./data/ring folder and delete everything in there, then everything works as expected.

Most of the examples in the book leveraged curl. However, I learn best if I tried to work the examples in another way. I tried the examples in Clojure, using ClojureWerkz's Welle. I liked ClojureWerkz's Welle wrappers to Riak and would probably use it if I had to develop on Java platform. I also wanted to work with Riak from .NET platform and hence I'm using F# to explore Riak. The following examples where done on Visual Studio 2010 with ASP.NET MVC 4 installed. This also gives me a chance to take the REST API in ASP.NET MVC4 for a spin.

Pinging RIAK

The very first example in the book is to ping the Riak cluster, here's how I implemented it in F#

open System.Net.Http
open System.Threading.Tasks

// My 3 instances of Riak
let riak1_url = "http://192.168.56.1:8098"
let riak2_url = "http://192.168.56.2:8098"
let riak3_url = "http://192.168.56.3:8098"

// Pick one to work with
let riakurl = riak1_url

let client = new HttpClient()

let ping () = client.GetAsync(sprintf "%s/ping" riakurl)

ping()

Running the ping, I would get the following response from F#:

val it : Task =
  System.Threading.Tasks.Task`1[System.Net.Http.HttpResponseMessage]
    {AsyncState = null;
     CreationOptions = None;
     Exception = null;
     Id = 4;
     IsCanceled = false;
     IsCompleted = false;
     IsFaulted = false;
     Result = StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
  Vary: Accept-Encoding
  Date: Wed, 30 Jan 2013 22:20:11 GMT
  Server: MochiWeb/1.1
  Server: WebMachine/1.9.0
  Server: (someone had painted it blue)
  Content-Length: 422
  Content-Type: application/json
};
     Status = RanToCompletion;}

Adding Content to RIAK

Let's start by putting some stuff into Riak with the following snippet of code

let put bucket key content =
    let put_url= sprintf "%s/riak/%s/%s" riakurl bucket key
    client.PutAsync(put_url,content)


let put_html bucket key html =
    let put_url= sprintf "%s/riak/%s/%s" riakurl bucket key
    let content = new StringContent(html)
    content.Headers.ContentType.MediaType <- "text/html"
    put bucket key content


"<html><body><h1>My latest favorite DB is RIAK</h1></body></html>"
|> put_html "favs" "db"

Running the above script gets the following response:

val it : Task =
  System.Threading.Tasks.Task`1[System.Net.Http.HttpResponseMessage]
    {AsyncState = null;
     CreationOptions = None;
     Exception = null;
     Id = 5;
     IsCanceled = false;
     IsCompleted = false;
     IsFaulted = false;
     Result = StatusCode: 204, ReasonPhrase: 'No Content', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
  Vary: Accept-Encoding
  Date: Fri, 01 Feb 2013 18:07:24 GMT
  Server: MochiWeb/1.1
  Server: WebMachine/1.9.0
  Server: (someone had painted it blue)
  Content-Length: 0
  Content-Type: text/html; charset=utf-8
};
     Status = RanToCompletion;}

We got the 204 code as explained in the book. To test that Riak has stored this new content, simply point to any of the Riak instances, (e.g. http://192.168.56.3:8098/riak/favs/db) with a browser and you should see the webpage that was put into the first Riak server.

Here's the sample code to put JSON data into Riak: 

// Simple utility to generate JSON - should really use a real JSON library
let tojson data = 
    data |> Seq.map (fun (k,v) -> sprintf "\"%s\" : \"%s\"" k v)
         |> Seq.reduce (sprintf "%s , %s")
         |> sprintf "{ %s }" 


let put_json bucket key jsondata =
    let content = new StringContent(tojson jsondata)
    content.Headers.ContentType.MediaType <- "application/json"
    put bucket key content

[("nickname","The Wonder Dog"); ("breed","German Shepherd")]
|> put_json "animals" "ace"

Again, you can check that it's stored in Riak by pointing the browser to: http://192.168.56.3:8098/riak/animals/ace and you should get back: 

{ "nickname" : "The Wonder Dog" , "breed" : "German Shepherd" }

Removing Content from RIAK

Here's a snippet of script to remove a content from Riak


let delete bucket key =
    let delete_url= sprintf "%s/riak/%s/%s" riakurl bucket key 
    client.DeleteAsync(delete_url)

delete "animals" "ace"

Getting Bucket Keys

To get all keys in a bucket

let get_keys bucket =
    let get_url = sprintf "%s/riak/%s?keys=true" riakurl bucket
    get_url |> client.GetStringAsync

let results = get_keys "animals"
printfn "%s" results.Result

The above script would return the following (reformatted for legibility purposes):

{"props":{"name":"animals",
          "allow_mult":false,
    "basic_quorum":false,
    "big_vclock":50,
    "chash_keyfun":{"mod":"riak_core_util",
                    "fun":"chash_std_keyfun"},
    "dw":"quorum",
    "last_write_wins":false,
    "linkfun":{"mod":"riak_kv_wm_link_walker",
               "fun":"mapreduce_linkfun"},
    "n_val":3,
    "notfound_ok":true,
    "old_vclock":86400,
    "postcommit":[],
    "pr":0,
    "precommit":[],
    "pw":0,
    "r":"quorum",
    "rw":"quorum",
    "small_vclock":50,
    "w":"quorum",
    "young_vclock":20},
    "keys":["ace","polly"]}

Retrieving Content from Riak

To retrieve content:

let get bucket key = 
    let get_url = sprintf "%s/riak/%s/%s/" riakurl bucket key
    get_url |> client.GetStringAsync

let results = get "animals" "ace"
printfn "%s" results.Result
Posted by John Liao at 6:00 AM
Labels: , , , ,

3 comments:

Dave Thomas said...

Have you tried the corrugated Iron .Net client?
http://corrugatediron.org

4/23/13, 10:56 AM
Anonymous said...

Nice little post. I like how it shows that there's a simple way of talking to Riak through the REST API.

Be careful not to oversimplify things though! Talking to a distributed system is a complex problem and there are many nuances that need to be catered for.

I'd be interested to hear your thoughts on our Riak library, the one that Dave Thomas has posted a link to. It's a C# lib at this point though we're pondering rewriting some parts of it using F#.

All the best.
OJ

4/23/13, 3:16 PM
John Liao said...

I have not tried Corrugated Iron. I did not know about this library and I will definitely take a look. I certainly understand that a distributed system is complex and in no way am I suggesting to use the REST API as the sole mechanism to interact with Riak. But having REST API as an option for interacting with Riak should be considered, especially for lightweight access mechanism.

4/24/13, 10:52 AM

Post a Comment

Subscribe to: Post Comments (Atom)