s3.getObject().createReadStream() : How to catch the error?

Question

I am trying to write a program to get a zip file from s3, unzip it, then upload it to S3. But I found two exceptions that I can not catch.

1.

Answer 1

You can listen to events (like error, data, finish) in the stream you are receiving back. Read more on events

function getObjectStream (filePath) {
  return s3.getObject({
    Bucket: bucket,
    Key: filePath
  }).createReadStream()
}

let readStream = getObjectStream('/path/to/file.zip')
readStream.on('error', function (error) {
  // Handle your error here.
})

Tested for "No Key" error.

it('should not be able to get stream of unavailable object', function (done) {
  let filePath = 'file_not_available.zip'

  let readStream = s3.getObjectStream(filePath)
  readStream.on('error', function (error) {
    expect(error instanceof Error).to.equal(true)
    expect(error.message).to.equal('The specified key does not exist.')
    done()
  })
})

Tested for success.

it('should be able to get stream of available object', function (done) {
  let filePath = 'test.zip'
  let receivedBytes = 0

  let readStream = s3.getObjectStream(filePath)
  readStream.on('error', function (error) {
    expect(error).to.equal(undefined)
  })
  readStream.on('data', function (data) {
    receivedBytes += data.length
  })
  readStream.on('finish', function () {
    expect(receivedBytes).to.equal(3774)
    done()
  })
})

Answer 2

If you'd like to catch the NoSuchKey error thrown by createReadStream you have 2 options:

1. Check if key exists before reading it.
2. Catch error from stream

First:

s3.getObjectMetadata(key)
  .promise()
  .then(() => {
    // This will not throw error anymore
    s3.getObject().createReadStream();
  })
  .catch(error => {
    if (error.statusCode === 404) {
      // Catching NoSuchKey
    }
  });

The only case when you won't catch error if file was deleted in a split second, between parsing response from getObjectMetadata and running createReadStream

Second:

s3.getObject().createReadStream().on('error', error => {
    // Catching NoSuchKey & StreamContentLengthMismatch
});

This is a more generic approach and will catch all other errors, like network problems.

Answer 3

You need to listen for the emitted error earlier. Your error handler is only looking for errors during the unzip part.

A simplified version of your script.

s3.getObject(params)
.createReadStream()
.on('error', (e) => {
  // handle aws s3 error from createReadStream
})
.pipe(unzip)
.on('data', (data) => {
  // retrieve data
})
.on('end', () => {
  // stream has ended
})
.on('error', (e) => {
  // handle error from unzip
});

This way, you do not need to make an additional call to AWS to find out if out if it exists.