November 1, 2012
Erlang was created to run on a variety of systems. Riak (written in Erlang) was created as a fault-tolerant distributed datastore, able to run on commodity hardware. Raspberry Pi is the culmination of these two points, brought to an absurd level: an embedded(ish), very inexpensive ($35) commodity computer. I thought it might be fun to create a Riak cluster on a set of Pis I had lying around.
Here’s what you’ll need to build your own RiakPi cluster:
- N Raspberry Pis
- N SD card, 4+ GB
- N CAT5 cables
- N 5V, 700-1200 mA micro USB powersource (the kind used by many cellphones)
- 1 really cheap, non-fancy USB keyboard
- 1 monitor/tv with HDMI input
- 1 HDMI cable
- 1 Hub, Switch, or Router
- 1 Laptop/Desktop with an SD card slot
If you only have 1 Pi, you can still install Riak, but clearly not create a cluster. For the rest of this post, I’ll assume N=3.
It may seem like a lot of parts. However, except for the RPis, you can find most of this stuff lying around RadioShack, or your local reclaimed electronics equipment store, fairly inexpensively.
Installing Riak on Raspberry Pi
Step 1: Install Raspbi
Get your laptop (or desktop) and insert your SD card, after you download the Raspbian image (the current release is “wheezy”).
Unzip the download, and you should have an
You’ll need to unmount the (probably) default partition your SD card started with, then write the image to the device. There are several detailed instructions for many operating systems on elinux.
I have a Mac running OSX, so I did the following. Be extra careful with the device names, or else you might just flash your hard drive if you get it wrong. Warning: be very, very careful here.
1. Find the device name of the SD’s partition
2. unmount that partition
sudo diskutil unmount /dev/disk1s1
3. Get the actual device name (which is just the disk number, minus the s1, prefixed with
r. In my case it was
4. The flag
if is the input file, and
of is the output device (the SD card)
sudo dd bs=1m if=~/Downloads/2012-08-16-wheezy-raspbian.img of=/dev/rdisk1
5. Eject the card
sudo diskutil eject /dev/rdisk1
Step 2: Plug everything in so it looks like this
Plug your shiny new SD card into your RPi. In fact, why not just plug everything in? Plug the power in last, or you’ll be racing against the boot up process trying to connect devices.
Simply plugging in the Pi will turn it on.
It should look something like this super hip Instagram photo.
Step 3: Boot up and setup
If everything booted up, you should see a Blue Screen of Life.
The first task I recommend is configuring your keyboard. Since Raspberry Pi is from the UK, Raspbian is set up with UK defaults (which, as an American, this came as quite a shock since everything is usually set up for me all the time).
The next thing you’ll want to do is expand the root partition to take up the full SD space. If you really wanted to make a production ready server you’d probably want to make a separate partition to store the Riak data… but if you wanted a production system you wouldn’t be installing on a Raspberry Pi anyway.
If you like, try upgrading
raspi-config. This isn’t necessary, but I like to keep current. You’ll certainly need an internet connection to do so.
Then reboot. You can just unplug it. At this point I wouldn’t worry about file corruption. If the thought bothers you, you can run
sudo shutdown now, then unplug.
When the server restarts, just select “finish” if the config screen launches again.
Step 4: Install
esl-erlang (R15B02, Raspian)
At this point, get ready to start waiting. RPi isn’t exactly a big iron machine, and SD cards are not fast storage, so the combination can be maddeningly sluggish. You might want to have a book handy.
Now that all of that housekeeping is out of the way, it’s time to get cracking on some Riak. The first thing you’ll need, however, is Erlang. Thankfully, the awesome Erlang Solutions folks created a Raspbi distro, which was way easier than the first two weeks when I tried to create my own build. So unless you’re masochistic, I suggest using theirs.
First, add the following line to your
/etc/apt/sources.list file. Luckily, vi is installed.
sudo vi /etc/apt/sources.list
deb http://binaries.erlang-solutions.com/debian wheezy contrib
Then add the Erlang Solutions public key for
wget -O - http://binaries.erlang-solutions.com/debian/erlang_solutions.asc | sudo apt-key add -
apt-get package list, and then install
sudo apt-get update
sudo apt-get install esl-erlang
Once erlang is installed, type
erl to test. I found that I received a segfault unless I restarted once after install.
If you are prompted for a password, the login/password is
Step 5: Install Riak from Source
With Erlang installed, now we move onto Riak proper. First thing is to download and extract Riak 1.2.1.
curl -O http://downloads.riak.com.s3-website-us-east-1.amazonaws.com/riak/CURRENT/riak-1.2.1.tar.gz
tar zxvf riak-1.2.1.tar.gz
Before we can actually install Riak using Rebar, we’ll first need to install
git. Rebar uses git to download Riak’s dependencies.
sudo apt-get install git
With git in place, run make a release version and let Rebar do its stuff.
Once your Riak release is installed, start up riak:
You can test that it’s started and receiving requests by curling
$ curl http://localhost:8098/ping
$ curl -XPUT http://localhost:8098/riak/hello/fr -d ‘Allo’
$ curl http://localhost:8098/riak/hello/fr
It’s worth nothing that the Raspberry Pi only has 256M of RAM. So feel free to tweak some of the default settings in
Step 6: Rinse and Repeat
At this point you should have Riak installed on a Raspberry Pi. To install on the remaining two, you can either clone the SD card image to SD cards, or you can install Riak from scratch onto those cards following the instructions above. Copying SD images can be slow, so I couldn’t quite recommend one over the other.
Build a Cluster
Once you have three working cards, let’s network the Pis into a Riak cluster.
Step 6: A Little Network
Before tackling this part, I’d stop Riak. We’ll fire it up when our network is functional.
You can plug your Pis into a router. They’re set up for DHCP by default so you should be good to go. You can largely skip this section.
However, if you’re cheap like me, you can plug them into a switch or hub, and do a bit of configuration.
/etc/network/interfaces file to have the following settings:
iface lo inet loopback
iface eth0 inet static
Then restart the network.
sudo /etc/init.d/networking restart
You should see your network lights flash off then on.
If you run
lo values should be what you expect.
If for some reason that didn’t work, try.
sudo ifconfig eth0 down
sudo ifconfig eth0 up
Then plug your keyboard and monitor into your other Pis, and repeat for the remaining two. But this time, give each card its own successive IP address (192.168.10.11, 192.168.10.12).
You should now be able to ping the other cards.
The LNK lights on your cards should blink for each
There are more creative ways to configure a network, obviously, but this was fine for my three little cards.
Step 7: Make Riak Nodes
Although each RPi has Riak installed, they are not configured with the new network settings which allow the Erlang VMs to communicate with each other.
Change to your riak install directory (
Replace all 127.0.0.1 with 192.168.10.10, or whichever your Pi’s IP.
Replace all 127.0.0.1 with 192.168.10.10/11/12.
Start riak with
bin/riak start and check that it’s running with
/bin/riak pong. If you have trouble getting Riak to start, you may have better luck by deleting your data directory.
Step 8: Cluster Time
Now each of your RPis is an official Riak node. It’s time to build a cluster!
Whatever RPi node you happen to be connected to, choose the two other nodes to join. Since I’m connected to
192.168.10.12, I typed the following:
./bin/riak-admin cluster join email@example.com
./bin/riak-admin cluster join firstname.lastname@example.org
You can see what percentage of the ring each node has by typing, which will give you a (not exactly evenly) divided 33% for each.
That’s it. Now you can throw some commands at one of the nodes, and the value should be available from any other node. Go ahead and try it.
I posted a value to the druplets buckets (a druplet is the little nodule on a raspberry), then listed the keys from a different IP.
curl -XPOST http://192.168.10.10:8098/riak/druplets -d 'YUM'
curl http://192.168.10.12:8098/riak/druplets?keys=true | json_pp
You should expect the new key to be added.
Note that cluster isn’t fit for any purpose in its current form. If I were serious about a production Pi cluster, I’d connect a better drive than an SD card (which could be anything), I’d certainly alter the
I’d love to hear what you do with this.