How to read a GZIP payload in Ruby Sinatra

问题

On a remote host I have a bash script sending a simple gzipped YAML file to my Ruby Sinatra endpoint:

#!/bin/bash

/bin/gzip -c /tmp/test.yaml > /tmp/test.gz

curl -i <hostname>:<port>/last_run_report -H "Content-Type: application/xml" -H "Content-Encoding: gzip" --data-binary @/tmp/test.gz

My sample Ruby app is:

require 'sinatra'
require 'zlib'
require 'stringio'

set :port, <port>
set :bind, "<ip>"

post '/last_run_report' do
  sio = StringIO.new(request.body.to_s)
  gz = Zlib::GzipReader.new(sio).read
  test_yaml = YAML.load(gz)
end

This gives me the following error:

Zlib::GzipFile::Error: not in gzip format

If I require 'base64' and change the endpoint definition to:

post '/last_run_report' do
  raw = Base64.decode64(request.body)
  sio = StringIO.new(raw)
  gz = Zlib::GzipReader.new(sio).read
  test_yaml = YAML.load(gz)
end

I get the following error:

NoMethodError: undefined method `unpack1' for #<StringIO:0x000055713e2d51b8>

I can't figure out if I'm sending the data wrong or reading it wrong. Please help.

回答1:

Assuming a YAML sample like the following:

martin:
    name: Martin D'vloper
    job: Developer
    skill: Elite

You don't need all the excess StringIO stuff. request.body is already a StringIO instance, so converting it to a string then converting that to a StringIO is unnecessary:

require 'sinatra'
require 'zlib'

post '/last_run_report' do
  gz = Zlib::GzipReader.new(request.body).read
  puts YAML.load(gz)
end

Now make your request:

curl -i localhost:4567/last_run_report -H "Content-Type: application/xml" -H "Content-Encoding: gzip" --data-binary @test.gz

And view the sinatra output:

== Sinatra (v2.0.4) has taken the stage on 4567 for development with backup from Thin
Thin web server (v1.7.2 codename Bachmanity)
Maximum connections set to 1024
Listening on localhost:4567, CTRL+C to stop
{"martin"=>{"name"=>"Martin D'vloper", "job"=>"Developer", "skill"=>"Elite"}}
::1 - - [14/Jan/2019:23:24:28 -0700] "POST /last_run_report HTTP/1.1" 200 - 0.0048

Note that puts has written {"martin"=>{"name"=>"Martin D'vloper", "job"=>"Developer", "skill"=>"Elite"}} to the console.

I should point out that in your example the following code does not work the way you expect:

sio = StringIO.new(request.body.to_s)

You expect to be able to call sio.read and get something like this:

\x1F\x8B\b\b\xA7z=\\\x00\x03test.yaml\x00SVp\xCCSH\xCD-\xC8\xC9\xAFLMU(JM\xCE/J\xE1\xCAM,*\xC9\xCC\xB3\xE2R\x00\x82\xBC\xC4\xDCT+\x05_\xB0\x88\x82\x8BzYN~Aj\x11X&+?\xC9J\xC1%\xB5,\x15!T\x9C\x9D\x99\x93c\xA5\xE0\x9A\x93Y\x92\n\x00\xFC\xA0\x83yZ\x00\x00\x00

What you in fact get is this:

#<StringIO:0x00007ffd8184bdf0>

Note that this is the literal string "#<StringIO:0x00007ffd8184bdf0>" and not a reference to a StringIO object, because that is what is returned when calling .to_s on a StringIO object like request.body, so any subsequent call to sio.read (which is implicit in the call to Zlib::GzipReader.new) will return that literal string and will not return the gzipped data that you expect it to return, which leads to the error Zlib::GzipFile::Error: not in gzip format.

When you want to return the string representation of a StringIO you should call .string or .read, not .to_s. With that in mind, the minimal change required to make your first example work is to change this: