Efficient way to pull data from second database?

Question

We have a primary database where all of our app resides.

But there\'s a second database (updated from an external source), that I\'d like to be able to connect to so

Answer 1

For simple scenarios, Rails can support this without any extra gems; simply define the database in database.yml:

other_db:
  adapter: mysql2
  encoding: utf8
  database: other_db
  username: user
  password: passwd
  host: 1.2.3.4
  port: 3306

Then in the model you want to use the other database add:

class Article < ActiveRecord::Base
  establish_connection(:other_db)
  self.table_name = 'other_db.articles'
end

And then you can perform your query:

Article.where("id > 1000")

=)

Answer 2

Firstly, we need to define the external database:

# config/database.yml
external:
  adapter: sqlite3
  database: db/external.sqlite3
  pool: 5
  timeout: 5000

For something like this, I prefer using a module that takes care of database connections.

# lib/database.rb
module Database
  def self.using(db, klass=ActiveRecord::Base)
    klass.establish_connection(db.to_sym)
    result = yield if block_given?
    klass.establish_connection(Rails.env.to_sym)
    result
  end
end

By passing a block to a module like this we can make sure we don't bleed our queries into a database where they don't belong or forget to revert our connection. Also, we default to ActiveRecord::Base so that we can work with any of our models. However, if we know we're only using one and want to isolate our model usage, we can pass it as the secondary parameter.

The resulting implementation can look something like this:

# app/models/user.rb
class User < ActiveRecord::Base
  def read_remote
    Database.using(:external) do
      account = Account.where(active: true)
      account.users
    end
  end
end

Now we have a nice and clean block we can use anywhere we like, use any models we want inside the block and always be operating on the extneral database. Once that block is finished, we know we're working back on our original database.

Answer 3

Copy/Paste:

For a single-master situation, you could define another database connection in database.yml for the read slave:

read_slave:
  adapter: postgresql
  database: read_only_production
  username: user
  password: pass
  host: read_slave_host

This database is backed by a module, which mirrors the ActiveRecord classes using this database connection:

require 'magic_multi_connections'
module ReadSlave
  establish_connection :read_slave
end

Now, all pre-existing models can be accessed through the read_slave connection by prefixing the model class with ReadSlave::.

# use the read-only connection
    @user = ReadSlave::User.find(params[:id])

    # write to the master (can't use @user.update_attributes because it would#
    try to write to the read slave)
    User.update(@user.id, :login => "new_login")