Can I serialize a ruby Digest::SHA1 instance object?

Question

G'day people,

I am re-implementing an existent custom file upload service in ruby (sinatra) with redis as a backing store.

Client calculates a SHA1 hash and initiates an upload uploads max 64K chunks until finished

Server appends chunks to file calculates SHA1 hash of complete file to verify correct receipt

Now, what I am hoping to do is use ruby (1.9.3) Digest::SHA1 << (update) operator on each chunk, (rather than having to read the ENTIRE file from scratch at the end). [Large files > 1GB].

Unfortunately Digest::SHA1 and Marshal.dump aren't compatible

1.9.3p125 :001 > require 'digest'
 => true 
1.9.3p125 :002 > $digest = Digest::SHA1.new
 => #<Digest::SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709> 
1.9.3p125 :003 > marshalled_digest = Marshal.dump($digest)
TypeError: no _dump_data is defined for class Digest::SHA1
    from (irb):3:in `dump'
    from (irb):3
    from /Users/rhodry/.rvm/rubies/ruby-1.9.3-p125/bin/irb:16:in `<main>'
1.9.3p125 :004 > 

Does anyone have any ideas on how to:

1. Get access to the underlying memory (manipulated in C) and store / restore an object like that?
2. Obtain an alternative implementation that would allow a similar use-case?

Thanks,

parameme

Update: gist:2280705 implements option 1 using ruby FFI - hope it is useful to someone else

Answer 1

Have you considered and are you able to send the SHA1's of the 64k chunks? There would be more checksum data, but you would know where things went wrong, and there would be no need to store internal state of the digest.