May 28, 2013
Shopzilla allows customers to easily find, compare, and buy anything on the web. They reach over 40 million customers each month, connecting them to 100 million products from tens of thousands of retailers. Riak powers both their Keyword and Scrapbook Data Services.
Previously, Shopzilla was using large installations of RDBMS as their primary data platform. This worked well for some use cases – however, it had unnecessary features and was expensive for some services, especially read-heavy use cases with large datasets. For these use cases, they wanted an open source solution that offered deployment on commodity hardware and support for high read and write throughput. With downtime having a direct impact on revenue, availability was a critical factor. Riak was evaluated alongside MongoDB, Redis, and HBase. Ultimately Riak not only fit all of their criteria, but also offered ease of management and operations, allowing their development team to focus on other areas of the business.
Shopzilla uses Riak to store data for its Keyword and Scrapbook Services. The Keyword Service serves metadata about primary keywords and requires real-time access. It stores over 1 billion keywords and can process tens of millions of new keywords at a time. They migrated this service over to a six-node Riak cluster that uses Varnish for caching. Riak is accessed via protocol buffers for high performance and uses the LevelDB backend, best suited to implementations with a very large number of keys. Under moderate load, they see 95% of traffic requests come in under 10 milliseconds.
The Scrapbook Service allows Shopzilla to aggregate product information from different web sources to add supplemental details on their product pages. This means they potentially need to store data equivalent to the scale of their product inventory. Since this product information is accessed via Product ID, it was a perfect fit for Riak’s key/value data model. To provide familiar ad-hoc SQL querying capabilities, they worked with their content team to aggregate and stage this data in an Oracle schema before bringing it into Riak. This allowed them to design a schema based on their needs that would not affect the speed in which Riak serves this data. Most of their online requests are served in well under 5 milliseconds.
Riak has been in production at Shopzilla for over a year now. According to Will Gage, Principal Software Architect at Shopzilla, “I haven’t had to worry about Riak since implementation because we haven’t had any significant problems with it. We’re confident we made the right choice with Riak. It works as it’s supposed to and its stability is great. We’ve watched it work through real-life network partitions under load and recover quickly with no intervention. In short, Riak lets us focus on things other than the database.”
For more information about why Shopzilla chose Riak, check out Gage’s talk from a recent Riak meetup in Santa Monica, below. To learn more about Riak, visit riak.com/riak.
