Speed up assets:precompile with Rails 3.1/3.2 Capistrano deployment

Question

My deployments are slow, they take at least 3 minutes. The slow Capistrano task during deploy is assets:precompile. This takes probably 99% of the total deploy time. How can

Answer 1

You can save your server effort for pre-compiling assets by doing the same (pre-compiling assets) on your local system. And just moving to server.

from = source.next_revision(current_revision) rescue nil      
if from.nil? || capture("cd #{latest_release} && #{source.local.log(from)} vendor/assets/ app/assets/ | wc -l").to_i > 0
  ln_assets    
  run_locally "rake assets:precompile"
  run_locally "cd public; tar -zcvf assets.tar.gz assets"
  top.upload "public/assets.tar.gz", "#{shared_path}", :via => :scp
  run "cd #{shared_path}; tar -zxvf assets.tar.gz"
  run_locally "rm public/assets.tar.gz"    
else
  run "ln -s #{shared_path}/assets #{latest_release}/public/assets"
  logger.info "Skipping asset pre-compilation because there were no asset changes"
end

Answer 2

I've just written a gem to solve this problem inside Rails, called turbo-sprockets-rails3. It speeds up your assets:precompile by only recompiling changed files, and only compiling once to generate all assets. It works out of the box for Capistrano, since your assets directory is shared between releases.

This is much more bulletproof than the solutions that use git log, since my patch analyzes the sources of your assets, even if they come from a gem. For example, if you update jquery-rails, a change will be detected for application.js, and only application.js will be recompiled.

Note that I'm also trying to get this patch merged into Rails 4.0.0, and possibly Rails 3.2.9 (see https://github.com/rails/sprockets-rails/pull/21). But for now, it would be awesome if you could help me test out the turbo-sprockets-rails3 gem, and let me know if you have any problems.

Answer 3

The OP explicitly asked for Capistrano, but in case you are deploying without a dedicated deploy tool (via bash script, Ansible playbook or similar), you may use the following steps to speed up your Rails deploys:

• Skip bundle installation
  bundle check returns 1 if there are gems to install (1 otherwise) so it's easy to skip bundle installation if not necessary.

• Skip asset precompilation
  Use git rev-parse HEAD before pulling changes and store the current version's SHA in a variable (say $previous_commit). Then pull changes and find out if assets have changed with the command git diff --name-only $previous_commit HEAD | grep -E "(app|lib|vendor)/assets". If this returns $1 you can safely skip asset precompilation (if you use release-based deploys you may want to copy your assets to your new release's directory).

• Skip database migrations
  If you are using MySQL, use the command mysql --user=USER --password=PASSWORD --batch --skip-column-names --execute="USE MYAPP; SELECT version FROM schema_migrations ORDER BY version DESC LIMIT 1;" from your applcation's root directory to get the name of the latest applied migration. Compare this to the output of the command ls db/migrate | tail -1 | cut -d '_' -f 1 (which returns the latest available migration). If they differ, you need to migrate. If not, you can skip database migrations.

Rails developers deploying with Ansible can further reduce their deploy time by turning facts gathering off if not needed (gather_facts: no) and use SSH pipelining (export ANSIBLE_SSH_PIPELINING=1).

If you want more detail, I recently wrote an article about this topic.

Answer 4

The solution that Ben Curtis propose does not work for me, because I do not copy the .git folder when deploying (slow and useless) :

set :scm, :git
set :deploy_via, :remote_cache
set :copy_exclude, ['.git']

I'm using the following snippet, whitout load 'deploy/assets'

task :assets, :roles => :app do
  run <<-EOF
    cd #{release_path} &&
    rm -rf public/assets &&
    mkdir -p #{shared_path}/assets &&
    ln -s #{shared_path}/assets public/assets &&
    export FROM=`[ -f #{current_path}/REVISION ] && (cat #{current_path}/REVISION | perl -pe 's/$/../')` &&
    export TO=`cat #{release_path}/REVISION` &&
    echo ${FROM}${TO} &&
    cd #{shared_path}/cached-copy &&
    git log ${FROM}${TO} -- app/assets vendor/assets | wc -l | egrep '^0$' ||
    (
      echo "Recompiling assets" &&
      cd #{release_path} &&
      source .rvmrc &&
      RAILS_ENV=production bundle exec rake assets:precompile --trace
    )
  EOF
end

Answer 5

There are times when I need to force skip asset precompile when deploying a fix asap. I use the following hack as a complement to other answers to do the job.

callback = callbacks[:after].find{|c| c.source == "deploy:assets:precompile" }
callbacks[:after].delete(callback)
after 'deploy:update_code', 'deploy:assets:precompile' unless fetch(:skip_assets, false)

This script will change the built-in asset-precompile hooking, so it will be called based on the skip_assets parameter. I can call cap deploy -S skip_assets=true to skip asset precompile completely.

Answer 6

The idea is that if you don't change your assets you don't need to recompile them each time:

This is the solution that Ben Curtis propose for a deployment with git:

 namespace :deploy do
      namespace :assets do
        task :precompile, :roles => :web, :except => { :no_release => true } do
          from = source.next_revision(current_revision)
          if releases.length <= 1 || capture("cd #{latest_release} && #{source.local.log(from)} vendor/assets/ app/assets/ | wc -l").to_i > 0
            run %Q{cd #{latest_release} && #{rake} RAILS_ENV=#{rails_env} #{asset_env} assets:precompile}
          else
            logger.info "Skipping asset pre-compilation because there were no asset changes"
          end
      end
    end
  end

Here is another approach based on asset age (https://gist.github.com/2784462) :

set :max_asset_age, 2 ## Set asset age in minutes to test modified date against.

after "deploy:finalize_update", "deploy:assets:determine_modified_assets", "deploy:assets:conditionally_precompile"

namespace :deploy do
  namespace :assets do

    desc "Figure out modified assets."
    task :determine_modified_assets, :roles => assets_role, :except => { :no_release => true } do
      set :updated_assets, capture("find #{latest_release}/app/assets -type d -name .git -prune -o -mmin -#{max_asset_age} -type f -print", :except => { :no_release => true }).split
    end

    desc "Remove callback for asset precompiling unless assets were updated in most recent git commit."
    task :conditionally_precompile, :roles => assets_role, :except => { :no_release => true } do
      if(updated_assets.empty?)
        callback = callbacks[:after].find{|c| c.source == "deploy:assets:precompile" }
        callbacks[:after].delete(callback)
        logger.info("Skipping asset precompiling, no updated assets.")
      else
        logger.info("#{updated_assets.length} updated assets. Will precompile.")
      end
    end

  end
end

If you prefer to precompile your assets locally you can use this task:

namespace :deploy do
  namespace :assets do
    desc 'Run the precompile task locally and rsync with shared'
    task :precompile, :roles => :web, :except => { :no_release => true } do
      from = source.next_revision(current_revision)
      if releases.length <= 1 || capture("cd #{latest_release} && #{source.local.log(from)} vendor/assets/ app/assets/ | wc -l").to_i > 0
        %x{bundle exec rake assets:precompile}
        %x{rsync --recursive --times --rsh=ssh --compress --human-readable --progress public/assets #{user}@#{host}:#{shared_path}}
        %x{bundle exec rake assets:clean}
      else
        logger.info 'Skipping asset pre-compilation because there were no asset changes'
      end
    end
  end
end 

Another interesting approach can be that of using a git hook. For example you can add this code to .git/hooks/pre-commit which checks if there are any differences in the assets files and eventually precompiles them and add them to the current commit.

#!/bin/bash

# source rvm and .rvmrc if present
[ -s "$HOME/.rvm/scripts/rvm" ] && . "$HOME/.rvm/scripts/rvm"
[ -s "$PWD/.rvmrc" ] && . "$PWD/.rvmrc"

# precompile assets if any have been updated
if git diff-index --name-only HEAD | egrep '^app/assets' >/dev/null ; then
  echo 'Precompiling assets...'
  rake assets:precompile:all RAILS_ENV=production RAILS_GROUPS=assets
  git add public/assets/*
fi

If you decide to use this approach you would probably need to change your config/environments/development.rb adding:

config.assets.prefix = '/assets_dev'

So that while in development you won't serve the precompiled assets.