问题
I am developing an RabbitMQ consumer with pika 0.10.0, and python 2.7 version.In my consumer client, I have a process that runs for a time period depending on input message. It can vary from 3 to 40 minutes. I do not want to disable heartbeat. Instead I am looking for some collback mechanism that can keep the connection alive until the delivery_tag is sent back. Is that possible?
Few link I got, all are suggesting to disable the heartbeat as workaround. But I do not want to disable it.
Ref:
Socket Error: 104 consuming messages with task that take a long time #753
BlockingConnection gets closed unexpectedly #734
Also, please let me know if any extra information is required. Thanks in advance.
回答1:
The only solution is to send heartbeat frames periodically.
When using a BlockingConnection
, you have to call the process_data_events function frequently enough (a time_limit
of zero is ok). When using a SelectConnection
or other async adapters, you have to ensure none of your processes are blocking, so that frames can be sent.
If your task is long running and you can't interrupt or split the process easily for some reason, you can run the task in another thread/process, and still have pika sending frames from the main thread. Just keep in mind that you should avoid using pika connections across threads (pika is not thread-safe at the moment).
回答2:
You will have to use a extra thread to handle this with BlockingConnection. I have write a simple Producer and Consumer with threading
module.
The basic idea is
Producer. Use 2 threads, when you publish in the main thread, the message is passed to a internal queue, then the communicate thread consume the message send it to rabbitmq.
Consumer. You call conn.start_consumming in the main thread, and pass all messages to an internal queue and use some workers consume and work on the messages.
The source code is here
There are still some conveats that I have not covered, but I think the basic idea might be helpful.
来源:https://stackoverflow.com/questions/46053349/keep-pika-blockingconnection-alive-without-disabling-heartbeat