问题
Hibernate Utils is creating the session factory along with Hikari configuration. Currently we are doing inside @Setup method of ParDo, but it opens way too many connections. So is there any good example to initialize connection pool per worker ?
回答1:
If you are using @Setup
method inside DoFn
to create a database connection keep in mind that Apache Beam would create connection pool per worker instance thread. This might result in a lot of database connections depending on the number of workers that spin up.
You can use a Singleton class inside a ParDo to create your connection pool and control the number of connections per worker. You can take a look here for a sample implementation
http://mail-archives.apache.org/mod_mbox/beam-issues/201905.mbox/%3CJIRA.13231789.1557156230000.210182.1557222420198@Atlassian.JIRA%3E
来源:https://stackoverflow.com/questions/59311710/google-cloud-dataflow-how-to-initialize-hikari-connection-pool-only-once-per-w