Ruby's pack and unpack explained

Question

Even after reading the standard documentation, I still can\'t understand how Ruby\'s Array#pack and String#unpack exactly work. Here is the example that\'s causing me the most t

Answer 1

The 'H' String directive for Array#pack says that array contents should be interpreted as nibbles of hex strings.

In the first example you've provided:

irb(main):002:0> chars.pack("H*")
=> "a"

you're telling to pack the first element of the array as if it were a sequence of nibbles (half bytes) of a hex string: 0x61 in this case that corresponds to the 'a' ASCII character.

In the second example:

irb(main):003:0> chars.pack("HHH")
=> "```"

you're telling to pack 3 elements of the array as if they were nibbles (the high part in this case): 0x60 corresponds to the '`' ASCII character. The low part or second nibble (0x01) "gets lost" due to missing '2' or '*' modifiers for "aTemplateString".

What you need is:

chars.pack('H*' * chars.size)

in order to pack all the nibbles of all the elements of the array as if they were hex strings.

The case of 'H2' * char.size only works fine if the array elements are representing 1 byte only hex strings.

It means that something like chars = ["6161", "6262", "6363"] is going to be incomplete:

2.1.5 :047 > chars = ["6161", "6262", "6363"]
 => ["6161", "6262", "6363"] 
2.1.5 :048 > chars.pack('H2' * chars.size)
 => "abc" 

while:

2.1.5 :049 > chars.pack('H*' * chars.size)
 => "aabbcc"

Answer 2

The Array#pack method is pretty arcane. Addressing question (2), I was able to get your example to work by doing this:

> ["61", "62", "63"].pack("H2H2H2")
=> "abc" 

See the Ruby documentation for a similar example. Here is a more general way to do it:

["61", "62", "63"].map {|s| [s].pack("H2") }.join

This is probably not the most efficient way to tackle your problem; I suspect there is a better way, but it would help to know what kind of input you are starting out with.

The #pack method is common to other languages, such as Perl. If Ruby's documentation does not help, you might consult analogous documentation elsewhere.

Answer 3

I expected both these operations to return the same output: "abc".

The easiest way to understand why your approach didn't work, is to simply start with what you are expecting:

"abc".unpack("H*")
# => ["616263"]

["616263"].pack("H*")
# => "abc"

So, it seems that Ruby expects your hex bytes in one long string instead of separate elements of an array. So the simplest answer to your original question would be this:

chars = ["61", "62", "63"]
[chars.join].pack("H*")
# => "abc"

This approach also seems to perform comparably well for large input:

require 'benchmark'

chars = ["61", "62", "63"] * 100000

Benchmark.bmbm do |bm|
  bm.report("join pack") do [chars.join].pack("H*") end
  bm.report("big pack") do chars.pack("H2" * chars.size) end
  bm.report("map pack") do chars.map{ |s| [s].pack("H2") }.join end
end

#                 user     system      total        real
# join pack   0.030000   0.000000   0.030000 (  0.025558)
# big pack    0.030000   0.000000   0.030000 (  0.027773)
# map pack    0.230000   0.010000   0.240000 (  0.241117)

Answer 4

We were working on a similar problem this morning. If the array size is unknown, you can use:

ary = ["61", "62", "63"]
ary.pack('H2' * ary.size)
=> "abc"

You can reverse it using:

str = "abc"
str.unpack('H2' * str.size)
=> ["61", "62", "63"]