How should I connect to a Redis instance from an AWS Lambda function?

Question

I\'m trying to build an API for a single-page web app using AWS Lambda and the Serverless Framework. I want to use Redis Cloud for storage, mostly for its combination of spe

Answer 1

I've now solved my own problem, and I hope I can be of help to someone experiencing this problem in the future.

There are two major considerations when connecting to a database like I did in the code above from a Lambda function:

1. Once context.succeed(), context.fail(), or context.done() is called, AWS may freeze any processes that haven't finished yet. This is what was causing AWS to log Connection closed on the second call to my API endpoint—the process was frozen just before Redis finished closing, then thawed on the next call, at which point it continued right where it left off, reporting that the connection was closed. Takeaway: if you want to close your database connection, make sure it's fully closed before you call one of those methods. You can do this by putting a callback in an event handler that's triggered by a connection close (.on('end'), in my case).
2. If you split your code into separate files and require them at the top of each file, like I did, Amazon will cache as many of those modules as possible in memory. If that's causing problems, try moving the require() calls inside a function instead of at the top of the file, then exporting that function. Those modules will then be re-imported whenever the function is run.

Here's my updated code. Note that I've also put my Redis configuration into a separate file, so I can import it into other Lambda functions without duplicating code.

The Event Handler

'use strict'

const lib = require('../lib/related')

module.exports.handler = function (event, context) {
  lib.respond(event, (err, res) => {
    if (err) {
      return context.fail(err)
    } else {
      return context.succeed(res)
    }
  })
}

Redis Configuration

module.exports = () => {
  const redis = require('redis')
  const jsonify = require('redis-jsonify')
  const redisOptions = {
    host: process.env.REDIS_URL,
    port: process.env.REDIS_PORT,
    password: process.env.REDIS_PASS
  }

  return jsonify(redis.createClient(redisOptions))
}

The Function

'use strict'

const rt = require('./ritetag')

module.exports.respond = function (event, callback) {
  const redis = require('./redis')()

  const tag = event.hashtag.replace(/^#/, '')
  const key = 'related:' + tag
  let error, response

  redis.on('end', () => {
    callback(error, response)
  })

  redis.on('ready', function () {
    redis.get(key, (err, res) => {
      if (err) {
        redis.quit(() => {
          error = err
        })
      } else {
        if (res) {
          // Tag is found in Redis, so send results directly.
          redis.quit(() => {
            response = res
          })
        } else {
          // Tag is not yet in Redis, so query Ritetag.
          rt.hashtagDirectory(tag, (err, res) => {
            if (err) {
              redis.quit(() => {
                error = err
              })
            } else {
              redis.set(key, res, (err) => {
                if (err) {
                  redis.quit(() => {
                    error = err
                  })
                } else {
                  redis.quit(() => {
                    response = res
                  })
                }
              })
            }
          })
        }
      }
    })
  })
}

This works exactly as it should—and it's blazing fast, too.