Vergelijk.nl, part of the Dutch Compare Group, provides a service that makes shopping easy and transparent by offering an independent comparison of products, shops and prices. Since 1999 it has been one of the most popular price and product comparison websites in the Netherlands. They have thousands of online shops connected and each shop sells hundreds to millions of products. Visitors of Vergelijk.nl can compare products and their prices within 3500 categories. In addition, visitors are able to review products and shops.
We were able to sit down with Max Vernimmen, IT-operations manager, to talk about his experiences with Riak.
Can you give us a quick overview on how you are using Riak KV?
We are using Riak KV to host the static content of our five product comparison websites. So Riak KV hosts data like .css, .js, and all our images. We’re running Riak KV in a production environment that is critical to our existence.
What did you use before Riak KV?
Before we used Riak KV, we used two file servers to host all this content which caused problems for several reasons. With two file servers we had the issue where some files would be on one server and not on the other, and the other way around. Updating files would take a lot of time, because of the millions of files in there. We had to use a directory structure several layers deep to keep the amount of files per directory manageable. But the most worrying part was the growth in the data, this was not a setup that we could scale.
What issues were key to your decision to update your system?
We wanted to solve scalability, consistency, and high availability challenges. And we were interested in an object store orientated system instead of a file based system.
How did you arrive at the decision to select Riak KV?
We started looking at open source solutions and quickly we found Riak KV to be the only solution that was mature, tested and able to solve our needs. We also looked at grid-fs and gluster-fs, but experience with gluster turned us away from it and grid-fs didn’t seem very well supported back then and had some other issues that wouldn’t work for us. The ease with which we could install and start using Riak KV was a good thing. Once we ran some performance tests and saw it was just as fast or faster than the file servers we had before, we were sold.
Can you tell us a bit about your deployment?
Currently we are running two cluster, a test cluster with three nodes and a six node production cluster. Each node has 1TB of disk storage. We now have about 1.2TB of data stored in the cluster. With a replication count of three and six nodes that means we have about 600GB stored per node.
We’re growing steadily at about 240GB per year, which is about 120GB per year per node. Of this data we serve about 130GB per day, but the most frequently accessed data is served from our caching layer. Since these are product images and products rarely disappear (they only go out of production and become less important) we almost don’t remove any content.
How’s Riak KV Handling this rapid growth?
The scalability of Riak KV is works very nicely. We are now able to take nodes offline without impacting our visitors, which means that we have also achieved the required high availability without any issues with consistency between nodes.
What advice would you give people who are currently exploring or evaluating a NoSQL or database solution?
When looking into a new database or storage system I recommend thinking about the CAP theorem and then select a shortlist of solutions that fit your needs. Once you have the shortlist, take a look at manageability: how easy is it to add a node, remove a node, is manual action needed when a component fails and returns, how to monitor, etc. Once we did that we noticed that the Riak products provided the lowest maintenance overhead in our shortlist.
What sorts of companies do you think would benefit from using Riak products?
Companies that
- want to stay on top
- need to increase their agility even further
- need to reduce costs
Using software like Riak, that reduces manual work and gives peace of mind, can help to do that.
