Mongo connection pooling for Storm topology

问题

To connect to a mongo database,in our non-storm applications we would create a singleton instance of the Mongo class and share it across the application

Would like to know what is the right way to implement a Mongo connection pooling in the storm framework?

Options we have tried :
We have a spout which reads from the Mongo database. we just put the singleton Mongo class inside the spout and initialise it in the open method? - But this approach made it impossible to share the Mongo instance to a bolt if needed.
would like to have some pointers to right way of doing it.

回答1:

Connection pooling within storm is a little bit more complicated since you need to consider a couple things:

1. You are running in a clustered environment -> multiple machines need their own connections
2. Each Storm worker runs in a separate JVM -> it is likely that executors are not running in the same worker... which means no sharing connections

So, don't worry too much about it. Continue to use the Singleton instance (like in @bridiver's answer) of your class within your Spouts and Bolts like normal. If they happen to share the same JVM, then they will share the same pool and you are good to go!

In regards to the initialization, here is what I do:

1. Pass configuration information into the spout/bolt via the constructor
2. Initialize your connections in the open (for spouts) or the prepare (for bolts) methods
3. Connections are not shared between the spout and bolt

回答2:

The lack of a topology initialization method can be problematic. What we have done is to call a method on a singleton that initializes the connection pool in the prepare method (to get the config) of any bolt that uses it. We use a synchronized method with a flag to ensure that the actual initialization is only performed once. There is a small performance penalty for calling the synchronized method, but in most cases it's not an issue.

Mongo.create_connection_pool(conf)

and the method

public static synchronized void create_connection_pool(Config conf) {
  if (connected)
     return;
  else {
    initialize connection...
    connected = true;
  }
}

来源：https://stackoverflow.com/questions/27521797/mongo-connection-pooling-for-storm-topology