Storing Data on Distributed RAM with MemCached

Table of contents:

Credit: Ben Bleything with Michael Granger

Problem

You need a lightweight, persistent storage space, and you have systems on your network that have unused RAM.

Solution

memcached provides a distributed in-memory cache. When used with a Ruby client library, it can be used to store almost any Ruby object. See the Discussion section below for more information, and details of where to get memcached.

In this example, well use Michael Grangers Ruby-MemCache library, available as the Ruby-MemCache gem.

Assume you have a memcached server running on the machine at IP address 10.0.1.201. You can use the memcache gem to access the cache as though it were a local hash. This Ruby code will store a string in the remote cache:

	require 
ubygems
	require memcache

	MC = MemCache.new 10.0.1.201

	MC[:test] = This string lives in memcached!

The string has been placed in your memcached with the key :test. You can fetch it from a different Ruby session:

	require 
ubygems
	require memcache

	MC = MemCache.new 10.0.1.201

	MC[:test] # => "This string lives in memcached!"

You can also place more complex objects in memcached. In fact, any object that can be serialized with Marshal.dump can be placed in memcached. Here we store and retrieve a hash:

	hash = {
	 :roses => are red,
	 :violets => are blue
	}

	MC[:my_hash] = hash
	MC[:my_hash][:roses] # => "are red"

Discussion

memcached was originally designed to alleviate pressure on the database servers for LiveJournal.com. For more information about how memcached can be used for this kind of purpose, see Recipe 16.17.

memcached provides a lightweight, distributed cache space where the cache is held in RAM. This makes the cache extremely fast, and it never blocks on disk I/O. When effectively deployed, memcached can significantly reduce the load on your database servers by farming out storage to unused RAM on other machines.

To start using memcached, youll need to download the server (see below). You can install it from source, or get it via most *nix packaging systems.

Next, find some machines on your network that have extra RAM. Install memcached on them, then start the daemon with this command:

	$ memcached -d -m 1024

This starts up a memcached instance with a 1024-megabyte memory cache (you can, of course, vary the cache size as appropriate for your hardware). If you run this command on the machine with IP address 10.0.1.201, you can then access it from other machines on your local network, as in the examples above.

memcached also supports more advanced functions, such as conditional sets and expiration times. You can also combine multiple machines into a single virtual cache. For more information about these possibilities, refer to the memcached documentation and to the documentation for the Ruby library that you e using.

Problem

Solution

Discussion

See Also