Keep pika BlockingConnection alive without disabling heartbeat

点点圈 提交于 2019-12-23 16:35:45

问题


I am developing an RabbitMQ consumer with pika 0.10.0, and python 2.7 version.In my consumer client, I have a process that runs for a time period depending on input message. It can vary from 3 to 40 minutes. I do not want to disable heartbeat. Instead I am looking for some collback mechanism that can keep the connection alive until the delivery_tag is sent back. Is that possible?

Few link I got, all are suggesting to disable the heartbeat as workaround. But I do not want to disable it.

Ref:

Socket Error: 104 consuming messages with task that take a long time #753

BlockingConnection gets closed unexpectedly #734

Also, please let me know if any extra information is required. Thanks in advance.


回答1:


The only solution is to send heartbeat frames periodically.

When using a BlockingConnection, you have to call the process_data_events function frequently enough (a time_limit of zero is ok). When using a SelectConnection or other async adapters, you have to ensure none of your processes are blocking, so that frames can be sent.

If your task is long running and you can't interrupt or split the process easily for some reason, you can run the task in another thread/process, and still have pika sending frames from the main thread. Just keep in mind that you should avoid using pika connections across threads (pika is not thread-safe at the moment).




回答2:


You will have to use a extra thread to handle this with BlockingConnection. I have write a simple Producer and Consumer with threading module.

The basic idea is

  1. Producer. Use 2 threads, when you publish in the main thread, the message is passed to a internal queue, then the communicate thread consume the message send it to rabbitmq.

  2. Consumer. You call conn.start_consumming in the main thread, and pass all messages to an internal queue and use some workers consume and work on the messages.

The source code is here

There are still some conveats that I have not covered, but I think the basic idea might be helpful.



来源:https://stackoverflow.com/questions/46053349/keep-pika-blockingconnection-alive-without-disabling-heartbeat

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!