RabbitMQ Pika and Django Channels websocket

问题

I am using Django Channels and RabbitMQ pika, for the first time. I am trying to consume from RabbitMQ queue. I am using Django Channels AsyncConsumer to group send it to everyone connected in the websocket.

User type 1 : Can create a task

User type 2 : Can accept the task.

Use case : When user type 1 creates the task it is published in the rabbitmq. When it is consumed from the queue, it has to be group-sent to frontend. And when the user type 2 accepts the task other instances of user type 2 cannot accept the same and we consume from the queue again and send the next task in the queue to everyone.

I have created the connection in a different thread using sync_to_async I am appending it to an in-memory list from the callback function. And whenever someone accepts I just pop it out of the list and acknowledge the queue.

class AcceptTaskConsumer(AsyncConsumer):
    body = [] #IN MEMORY LIST 
    delivery = {} #To store ack delivery_tag 


    async def websocket_connect(self, event):
        print("AcceptTaskConsumer connected", event)
        AcceptTaskConsumer.get_task() #STARTS Queue listener in new thread
        self.room_group_name = "user_type_2"
        await self.channel_layer.group_add(
            self.room_group_name,
            self.channel_name
        )

        await self.send({
            "type": "websocket.accept"
        })

    async def websocket_receive(self, event):
        if event["text"] == "Hi": #If connecting first time
            if AcceptTaskConsumer.body:
                await self.channel_layer.group_send(
                    self.room_group_name,
                    {
                        "type": "message",
                        "text": AcceptTaskConsumer.body[0]["body"]
                    }
                )
            else:
                await self.channel_layer.group_send(
                    self.room_group_name,
                    {
                        "type": "message",
                        "text": "No New Tasks"
                    }
                )

        else: #When someone accepts a task-> ack and send next task in queue
            print(json.loads(event["text"])["id"])
            AcceptTaskConsumer.channel.basic_ack(delivery_tag=AcceptTaskConsumer.delivery[json.loads(event["text"])["id"]])
            AcceptTaskConsumer.delivery.pop(json.loads(event["text"])["id"])
            AcceptTaskConsumer.body.pop(0)
            await self.channel_layer.group_send(
                self.room_group_name,
                {
                    "type": "message",
                    "text": "No New Tasks"
                }
            )

            if AcceptTaskConsumer.body:
                await self.channel_layer.group_send(
                    self.room_group_name,
                    {
                        "type": "message",
                        "text": AcceptTaskConsumer.body[0]["body"]
                    }
                )

    async def message(self, event):
        await self.send({
            "type": "websocket.send",
            "text": event["text"]
        })

    @classmethod
    @sync_to_async
    def get_task(cls): #pika consumer
        cls.connection = pika.BlockingConnection(
            pika.ConnectionParameters(host='localhost'))
        cls.channel = cls.connection.channel()

        cls.channel.queue_declare(queue='task_', arguments={"x-max-priority": 3})

        cls.channel.basic_consume(
            queue='task_', on_message_callback=AcceptTaskConsumer.callback, auto_ack=False)
        cls.channel.start_consuming()

    @classmethod
    def callback(cls, ch, method, properties, body):
        task_obj = {"body": json.dumps(body.decode("utf-8")),
                    "delivery_tag": method.delivery_tag}
        AcceptTaskConsumer.body.append(task_obj)
        AcceptTaskConsumer.delivery[json.loads(json.loads(task_obj["body"]))["id"]] = method.delivery_tag
        cls.channel.stop_consuming()

    async def websocket_disconnect(self, event):
        print(event)
        await self.send({
            "type": "websocket.close"
        })

        await self.channel_layer.group_discard(
            self.room_group_name,
            self.channel_name
        )

I am pretty sure this is not the right way to do it, because it's not working as expected

I run into frequent errors like.

• 39 of 169 channels over capacity in group delivery
• pika.exceptions.StreamLostError: Stream connection lost: BrokenPipeError(32, 'Broken pipe')

I tried running the queue listener like this answer as well. Nothing working. Any one experienced has any thoughts about this? Is there a better way to approach this problem.?

回答1:

you should move the rabitMQ cosumering logic out of the websocket consumer.

Just have a django command that runs the Rabbit Consumer, that consumer can take messages from RabbitMQ and then use send_group to send them over groups to channels.

if your django command you will need to call send_group see https://channels.readthedocs.io/en/latest/topics/channel_layers.html#using-outside-of-consumers

from channels.layers import get_channel_layer

channel_layer = get_channel_layer()

async_to_sync(
    channel_layer.group_send
)(
    "user_type_2",
    {"type": "message", "msg": 123}
)

---

Then in the websocket consumer you should subscribe to the groups that the user wants/has permition to get.

来源：https://stackoverflow.com/questions/61651947/rabbitmq-pika-and-django-channels-websocket