mongoid query caching

Question

Rails\' ActiveRecord has a feature called Query Caching (ActiveRecord::QueryCache) which saves the result of SQL query for the life-span of a request. While I\'m not very fa

Answer 1

OK, Mongoid 4 supports QueryCache middleware.

Just add middleware in application.rb

config.middleware.use "Mongoid::QueryCache::Middleware"

And then profit:

  MOPED: 127.0.0.1:27017 QUERY        database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dabb6d61631e21d70000')}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.4397ms
  MOPED: 127.0.0.1:27017 QUERY        database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dacf6d61631e21dc0000')}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.4590ms
  QUERY CACHE                         database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564c9596d61631e21d30000')}, "$orderby"=>{:_id=>1}}
  QUERY CACHE                         database=XXX collection=page_variants selector={"$query"=>{"_id"=>BSON::ObjectId('5564dabb6d61631e21d70000')}, "$orderby"=>{:_id=>1}}

---

Source:

Mongoid changelog

https://github.com/mongoid/mongoid/blob/master/CHANGELOG.md#new-features-2

3410 Mongoid now has a query cache that can be used as a middleware in Rack applications. (Arthur Neves)

For Rails:

config.middleware.use(Mongoid::QueryCache::Middleware)

Answer 2

Mongoid 4.0+ now has a QueryCaching module: http://www.rubydoc.info/github/mongoid/mongoid/Mongoid/QueryCache

You can use it on finds by wrapping your lookups like so:

QueryCache.cache { MyCollection.find("xyz") }

Answer 3

Mongoid has caching, described under http://mongoid.org/en/mongoid/docs/extras.html

Also MongoDB itself has caching ability: http://www.mongodb.org/display/DOCS/Caching

The mongoid caching extra knows 2 different cases: Caching of all queries of a model or caching of a query.

Mongoid caching seems to work slightly different: it looks like mongoid delegates caching to mongodb. (In the sources of mongoid I only can find option settings for caching but no cache module.)

Finally would say, there is no real difference in the caching in general -- in memory is in fact in memory! No matter if it's in the app or in the database.

I don't prefer to implement an extra caching algorithm, because this seems to be redundant and a RAM killer.

BTW: If your really want to cache results in-app you could try Rails.cache or another cache gem as a workaround.

Answer 4

The other answer is obviously wrong. Not only mongoid or mongo driver doesn't cache the query, even if mongo would - it still might be on other machine across the network.

My solution was to wrap the receive_message in Mongo::Connection. Pros: one definite place Cons: deserialization still takes place

require 'mongo'
module Mongo
  class Connection
    module QueryCache
      extend ActiveSupport::Concern

      module InstanceMethods

        # Enable the selector cache within the block.
        def cache
          @query_cache ||= {}
          old, @query_cache_enabled = @query_cache_enabled, true
          yield
        ensure
          clear_query_cache
          @query_cache_enabled = old
        end

        # Disable the selector cache within the block.
        def uncached
          old, @query_cache_enabled = @query_cache_enabled, false
          yield
        ensure
          @query_cache_enabled = old
        end

        def clear_query_cache
          @query_cache.clear
        end

        def cache_receive_message(operation, message)
          @query_cache[operation] ||= {}
          key = message.to_s.hash
          log = "[MONGO] CACHE %s"
          if entry = @query_cache[operation][key]
            Mongoid.logger.debug log % 'HIT'
            entry
          else
            Mongoid.logger.debug log % 'MISS'
            @query_cache[operation][key] = yield
          end
        end

        def receive_message_with_cache(operation, message, log_message=nil, socket=nil, command=false)
          if query_cache_enabled
            cache_receive_message(operation, message) do
              receive_message_without_cache(operation, message, log_message, socket, command)
            end
          else
            receive_message_without_cache(operation, message, log_message, socket, command)
          end
        end
      end # module InstanceMethods

      included do
        alias_method_chain :receive_message, :cache
        attr_reader :query_cache, :query_cache_enabled
      end
    end # module QueryCache
  end # class Connection
end

Mongo::Connection.send(:include, Mongo::Connection::QueryCache)