rails caching with redis – invalidation done right

in my last article about using redis as rails cache i used the redis ‘keys’ function to invalidate cache.
as my readers noted, and i experienced myself, this is not a good idea if you plan on storing a few million keys in your redis-db.

so today we’ll look at how to do this invalidation thing in a nicer and faster way

as an example we’ll use the same ‘thingy’ model as the last time.

so how do we tackle this cache invalidation issue correctly. first off we have the advantage of rails nice MVC convention, which means usually our cache belongs to an instance of specific model class or a model class in general.

this means we should store which urls were cached in redis sets named after the model they belong to.

if we want to be able to invalidate on instance and class level i suggest following structure:

redis sets:
    model_name: model_ids
    model_name_id: cached_urls

so how to we generate this? we’ll add following code to our application_controller.rb

def save_cache_to_redis
  #just like before
  $redis_cache.setex( request.request_uri, @cache_lifetime, response.body )
  #now comes the new magic
  $redis_cache.sadd("#{@model_name.downcase}_instances_collection", @model_id)
  $redis_cache.sadd("#{@model_name.downcase}_#{@model_id}_urls_collection", request.request_uri)
end

so why not automagically generate the model_name and the model_id from the controller assuming we wrote the perfect restful application? because the world isn’t perfect and most of the time you would end up writing a lot of code just to make this ‘magic’ work. also nested routes and non restful stuff would be really complicated.
also you may have noticed i used some extra descriptive names for the sets. this is simply to avoid collisions in your redis key space (in case your model names are a very common).
so obviously we also need to do some work in the controllers we want to be cached.
let’s look at thingies_controller.rb

after_filter :save_cache_to_redis, :except => [:create, :destroy, :update]
...
def index
  #just like last time
  @cache_lifetime = 86400 #after 1 day the request will hit the controller again
  #but here comes the magic
  @model_name = "Thingy"
  @model_id = "index"
  ...
end

def show
  #and now for a single instance
  @cache_lifetime = 86400 #of course this should be made DRY
  @model_name = "Thingy" #same DRY todo as above
  @model_id = params[:id] #you may wanna you smth. santized here ;)
  ...
end

now you may say ‘i saw what you did there’…if you are clever you noticed i killed 3 flies with this…
let’s see the model in order to understand better.
here is thingy.rb

class Thingy < ActiveRecord::Base
  after_save :invalidate_index_cache, :invalidate_instance_cache
  after_destroy :invalidate_index_cache, :invalidate_instance_cache
  ...
  #you could refactor this to be a class method
  def invalidate_index_cache
   $redis_cache.smembers("thingy_index_urls_collection").each do |url|
     $redis_cache.del(url)
     $redis_cache.srem("thingy_index_urls_collection", url)
   end
  end
 
  def invalidate_instance_cache
   $redis_cache.smembers("thingy_#{self.id}_urls_collection").each do |url|
     $redis_cache.del(url)
     $redis_cache.srem("thingy_#{self.id}_urls_collection", url)
   end
  end

  #in case you ever want to wipe all the caches for this model
  def self.invalidate_class_cache
   $redis_cache.smembers("thingy_instances_collection").each do |instance_id|
     $redis_cache.smembers("thingy_#{instance_id}_urls_collection").each do |url|
       $redis_cache.del(url)
     end
     $redis_cache.del("thingy_#{instance_id}_urls_collection")
     $redis_cache.srem("thingy_instances_collection", instance_id)
   end
  end
  
end

we can now easily invalidate cache for the index, an instance or the whole class without using the slow redis 'keys' command.

that's it. i hope those of you who tried using my last idea to cache millions of pages can forgive me now ;)

next time i'll talk about automatic cache generation

so till then, as always

have fun