sidekiq does not process jobs, jobs stuck in enqueue

问题

I have a simple application that works with the twitter stream api, sidekiq, unicorn and sinatra. All works well, instead of... the job are not being processed at all. They are stuck in Enqueued (And I know that from the sidekiq web UI).

This is my code:

Procfile:

sidekiq: bundle exec sidekiq -C config/sidekiq.yml -e development -r ./lib/tweet_streamer.rb
unicorn: bundle exec unicorn -c config/unicorn.rb
redis: redis-stable/src/redis-server

config/unicorn.rb:

listen 5000, :tcp_nopush => true
timeout 30
preload_app true
GC.respond_to?(:copy_on_write_friendly=) and
  GC.copy_on_write_friendly = true

before_fork do |server, worker|
end

after_fork do |server, worker|
end

the tweet_streamer.rb:

require_relative "../config/tweetstream"
require_relative "workers"

class TweetStreamer

  client = TweetStream::Client.new
  client.on_enhance_your_calm do |s|
    puts "on_enhance_your_calm #{s}"
  end

  puts "onstream"

  client.userstream do |status|
    ##
    ## TODO: Some kind of log
    ##
    puts "start process"
    # TweetStatusWorker.perform_async(status.to_hash)
    Sidekiq::Client.push('class' => TweetStatusWorker, 'args' => ["a"])
    puts "process started!"
  end

end

the workers.rb

require_relative "../config/sidekiq"
require_relative 'workers/tweet_status_worker'

the config/sidekiq.rb:

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { :size => 1 }
end

Sidekiq.configure_server do |config|
  config.redis = { :size => 6 }
end

the worker:

class TweetStatusWorker
  include Sidekiq::Worker
  def perform(status)
    logger.warn "I'm working on it!"
  end
end

My sidekiq.yml:

 ---
:pidfile: tmp/pids/sidekiq.pid
development:
  :verbose: true
  :logfile: log/sidekiq_development.log

My config.ru:

require "rubygems"
require "sinatra"

Bundler.require

require File.expand_path '../twitter_module.rb', __FILE__

require 'sidekiq/web'

run Rack::URLMap.new('/' => TwitterModule, '/sidekiq' => Sidekiq::Web)

My app class:

require 'sinatra/base'

class TwitterModule < Sinatra::Base
end

This is the log that I received when I start the application and I make a tweet for start a process and all seems to be fine..

17:36:31 sidekiq.1 | started with pid 65599
17:36:31 unicorn.1 | started with pid 65600
17:36:31 redis.1   | started with pid 65601
17:36:31 redis.1   | [65601] 14 Oct 17:36:31.717 # Warning: no config file specified, using the default config. In order to specify a config file use redis-stable/src/redis-server /path/to/redis.conf
17:36:31 redis.1   | [65601] 14 Oct 17:36:31.718 * Max number of open files set to 10032
17:36:31 redis.1   |                 _._                                                  
17:36:31 redis.1   |            _.-``__ ''-._                                             
17:36:31 redis.1   |       _.-``    `.  `_.  ''-._           Redis 2.6.16 (f132ada8/1) 64 bit
17:36:31 redis.1   |   .-`` .-```.  ```\/    _.,_ ''-._                                   
17:36:31 redis.1   |  (    '      ,       .-`  | `,    )     Running in stand alone mode
17:36:31 redis.1   |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
17:36:31 redis.1   |  |    `-._   `._    /     _.-'    |     PID: 65601
17:36:31 redis.1   |   `-._    `-._  `-./  _.-'    _.-'                                   
17:36:31 redis.1   |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
17:36:31 redis.1   |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
17:36:31 redis.1   |   `-._    `-._`-.__.-'_.-'    _.-'                                   
17:36:31 redis.1   |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
17:36:31 redis.1   |  |    `-._`-._        _.-'_.-'    |                                  
17:36:31 redis.1   |   `-._    `-._`-.__.-'_.-'    _.-'                                   
17:36:31 redis.1   |       `-._    `-.__.-'    _.-'                                       
17:36:31 redis.1   |           `-._        _.-'                                           
17:36:31 redis.1   |               `-.__.-'                                               
17:36:31 redis.1   | 
17:36:31 redis.1   | [65601] 14 Oct 17:36:31.719 # Server started, Redis version 2.6.16
17:36:31 redis.1   | [65601] 14 Oct 17:36:31.728 * DB loaded from disk: 0.010 seconds
17:36:31 redis.1   | [65601] 14 Oct 17:36:31.728 * The server is now ready to accept connections on port 6379
17:36:33 unicorn.1 | I, [2013-10-14T17:36:33.208428 #65600]  INFO -- : Refreshing Gem list
17:36:33 unicorn.1 | I, [2013-10-14T17:36:33.813454 #65600]  INFO -- : listening on addr=0.0.0.0:5000 fd=9
17:36:33 unicorn.1 | I, [2013-10-14T17:36:33.814790 #65600]  INFO -- : master process ready
17:36:33 unicorn.1 | I, [2013-10-14T17:36:33.816138 #65607]  INFO -- : worker=0 ready
17:36:33 sidekiq.1 | onstream
17:36:43 sidekiq.1 | start process
17:36:43 sidekiq.1 | process started!

nothing in the sidekiq log obviously:

2013-10-14T16:36:43Z 65599 TID-ouo5je95k INFO: Booting Sidekiq 2.15.1 using redis://localhost:6379/0 with options {:size=>6}
2013-10-14T16:37:32Z 65599 TID-ouo5je95k DEBUG: Terminating 4 actors...

UPDATE:

Ok I found it. Is the streamer the problem. If I run the job outside of the streamer block, this works well. Any Idea about why this append and how I can solve it?

来源：https://stackoverflow.com/questions/19364978/sidekiq-does-not-process-jobs-jobs-stuck-in-enqueue

标签

ruby

Redis

unicorn

sidekiq