Heroku file upload problem

Question

I've been having a problem uploading a CSV file to Heroku and processing it. It works fine in my local environment. Just do be clear, I don't need to save the file on Heroku, just access it during the request in order to convert it into a string for processing and importing into the DB.

What I want to do is:

1. Upload CSV file
2. Strip out the header block, depending on which network the report is from
3. Read the CSV data into the DB. This step works fine.

Controller code:

  def create
    @account = Account.find(params[:report][:account_id])
    @file = params[:report][:file].read
    # logger.info file.inspect
    case @account.provider
    when "Microsoft AdCenter" then @file.gsub!(/\A(.*)\n\n/im, "")
    when "Google AdWords" then @file.gsub!(/\A(.*)\n/i, "")
    else
      raise "Invalid PPC report format"
    end
  end

Here's the stack trace:

Processing ImportController#create (for XX.182.6.XXX at 2010-09-11 09:19:01) [POST]
  Parameters: {"commit"=>"Upload", "action"=>"create", "authenticity_token"=>"XXXXXwoFpvRO3vN8XVXRDg8rikFsj2TFTW7mrcTgg=", "controller"=>"import", "report"=>{"account_id"=>"1", "file"=>#<File:/home/slugs/126077_0657264_9a92/mnt/tmp/RackMultipart.9845.0>}}

NoMethodError (private method `gsub!' called for #<Tempfile:0x2b8ccb63ece0>):
  /usr/local/lib/ruby/1.8/delegate.rb:270:in `method_missing'
  app/controllers/import_controller.rb:15:in `create'
  warden (0.10.7) lib/warden/manager.rb:35:in `call'
  warden (0.10.7) lib/warden/manager.rb:34:in `catch'
  warden (0.10.7) lib/warden/manager.rb:34:in `call'
  /home/heroku_rack/lib/static_assets.rb:9:in `call'
  /home/heroku_rack/lib/last_access.rb:25:in `call'
  /home/heroku_rack/lib/date_header.rb:14:in `call'
  thin (1.0.1) lib/thin/connection.rb:80:in `pre_process'
  thin (1.0.1) lib/thin/connection.rb:78:in `catch'
  thin (1.0.1) lib/thin/connection.rb:78:in `pre_process'
  thin (1.0.1) lib/thin/connection.rb:57:in `process'
  thin (1.0.1) lib/thin/connection.rb:42:in `receive_data'
  eventmachine (0.12.6) lib/eventmachine.rb:240:in `run_machine'
  eventmachine (0.12.6) lib/eventmachine.rb:240:in `run'
  thin (1.0.1) lib/thin/backends/base.rb:57:in `start'
  thin (1.0.1) lib/thin/server.rb:150:in `start'
  thin (1.0.1) lib/thin/controllers/controller.rb:80:in `start'
  thin (1.0.1) lib/thin/runner.rb:173:in `send'
  thin (1.0.1) lib/thin/runner.rb:173:in `run_command'
  thin (1.0.1) lib/thin/runner.rb:139:in `run!'
  thin (1.0.1) bin/thin:6
  /usr/local/bin/thin:20:in `load'
  /usr/local/bin/thin:20

Rendering /disk1/home/slugs/126077_0657264_9a92/mnt/public/500.html (500 Internal Server Error)

Anyone know why it works just fine locally but then produces that error on Heroku?

Thanks!

Answer 1

Based on Avishal's answer I use this with Rails 3:

@file = IO.read(params[:report].tempfile.path)

Answer 2

Not entirely true, actually. It turns out that if you upload a file to Heroku, you can access it for the duration of the request via the Tempfile class. I was able to read the tempfile into a string and then process it as needed (it's a CSV):

@file = IO.read(params[:report][:file].path)

Works fine for text/csv files, but I guess if you want to do more than basic text manipulation you have to go with S3 as Joost suggests.

Answer 3

Are you aware of this?

http://devcenter.heroku.com/categories/platform-constraints

Answer 4

I am still suck with this problem. I tried the solution given here to be able to do a csv file upload and then parse it to fill my DB through acive record. I have the following code:

file = IO.read(params[:file].tempfile.path)
    FasterCSV.new(file, :headers => true).each do |row|
         # my parsing logic
    end

The code just works perfect in the local but does not work at all in Heruko. I get error and all that I could see in the heruko logs is this:

Started POST "/projects/1/upload_pivotal_csv" for 122.172.25.106 at 2012-03-23 07:45:59 +0000 2012-03-23T07:46:00+00:00 app[web.1]: 2012-03-23T07:46:00+00:00 app[web.1]: NotImplementedError (Please switch to Ruby 1.9's standard CSV library. It's FasterCSV plus support for Ruby 1.9's m17n encoding engine.): 2012-03-23T07:46:00+00:00 app[web.1]: app/controllers/projects_controller.rb:17:in `upload_pivotal_csv' 2012-03-23T07:46:00+00:00 app[web.1]: 2012-03-23T07:46:00+00:00 app[web.1]: 2012-03-23T07:46:00+00:00 heroku[router]: POST castletrack.herokuapp.com/projects/1/upload_pivotal_csv dyno=web.1 queue=0 wait=0ms service=783ms status=500 bytes=728 2012-03-23T07:46:00+00:00 app[web.1]: Processing by ProjectsController#upload_pivotal_csv as HTML 2012-03-23T07:46:00+00:00 app[web.1]: Parameters: {"utf8"=>"✓", "authenticity_token"=>"sBmRWpGP3q9Hu7O2cMlmnGTByaTXValxYHw5+cFoSw0=", "file"=>#>, "commit"=>"Import", "id"=>"1"} 2012-03-23T07:46:00+00:00 app[web.1]: Completed in 406ms

I am sure it has something to do with the ruby version. I am using ruby 1.8.7 locally and its a rails 3 application. I am not sure what ruby version is there on heruko. But I can tell that I am using ceder stack.

Answer 5

Yep. the file system is read-only. You can store your files on S3.