using lambda functions to unzip archives in S3 is really sloooooow

Question

My company is uploading large archive files to S3, and now wants them to be unzipped on S3. I wrote a lambda function based on unzip, triggered by arrival of a file to the xxx-z

Answer 1

I suspect that the unzip module you are using is a JavaScript implementation that allows you to unzip zip files - which is very slow.

I recommend using gzip to compress the files and using the internal zlib library that is C compiled and should provide much better performance.

In case you choose to stick with zip, you could contact amazon support and ask to increase the 60 seconds limit on your lambda function.