Django Channels Background Worker Instances

问题

I am working on a Django Channels-2 project, where channel layer is used to offload information from browsers to a consumer who analyses it and send the results back to the Browsers via groups mechanism.

The data comes from pusher via a pusher-client, and is therefore the same for many client browsers. My first took was to set-up the following infrastructure:

layout 1:
(browser1: Pusher-client, websocket; ... browserN: Pusher-client, websocket)
<---->
WebsocketConsumer(member of group; sends to channel, recieves from group)
<---->
BackgroundWorker(listens on channel; sends to group)

This layout works fine and it offloads the need to burden my server with the pusher connection, which is left to individual browsers. But this also means that I propagate multiple copies of the same event down to my Worker. I understand there are solutions to this problem, but they seemed a bit awkward to me. Also, this implies a lot of JavaScript-ing in the browsers. Even more so since only one websocket connection is permitted per browser tab and I need to traffic everything trough the same socket. Therefore I use the following set-up now:

layout 2:
(browser1: websocket; ... browserN: websocket)
<---->
WebsocketConsumer(member of group; sends to channel, recieves from group)
<---->
BackgroundWorker(pusher-client; listens on channel; sends to group)

In this layout, the browser notifies the consumers It wants results of the data, and if the background worker hasn't started yet it is spawned and starts pusher-client and on every event received analyzes the data and sends the result to group and hence back to browsers. This all works very nice and fine.

THE QUESTION:
Now, when I run

./manage.py runworker *channel-name*

The worker consumer isn't started until it is actually needed. And when the first message arrives to the channel it listens on, it is fired up and enters the event-loop of the pusher-client. Subsequent calls do not spawn additional workers, it would seems, and I have a flag if the pusher is already started, so I have only one connection to the pusher active.

Q1: Can I count on the background consumer to stay alive after it started or is there a timeout or something if there is no new message to be read in the channel que?

Q2: Is there a better/more reliable layout for my operations?

I had also an external websocket server version where pusher was active and results made available, but I didn't see how to elegantly integrate it into the django channels project although it worker perfectly fine. Is this nonetheless a better approach and utilizing unix-sockets?

Thank you very much for your input!

[EDIT 1]

I have found in the Channels-1.x series docs an entry in the FAQ section, that it is possible to communicate with the Django channels app also externally via the channel layer directly. So this is the approach I'm testing right now and it works beautifully. So I am currently doing it this way. Since both Q1 and Q2 are still valid and of interest to me, I will keep the question alive.

A quick overview of how it is being done at the moment for interested is being shown in the remainder of the edit:

External.py

from <proj-name>.asgi import channel_layer
from asgiref.sync import async_to_sync
import pysher
...
# External program logic #
...
# Channel layer communication -> send_group #
async_to_sync(channel_layer.group_send)(
        <group>,
        {
            'type': "<type>",
            'data': <dict>,
        }
    )

in asgi.py file of the Django Channels Project

import os
import django
from channels.routing import get_default_application
from channels.layers import get_channel_layer

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "<proj-name>.settings")
django.setup()
application = get_default_application()

# || the important part for us || #
# vv                           vv #
channel_layer = get_channel_layer()

I'm running Django Channels 2.x series, which differs slightly from 1.x series. This setup eliminates the need for the background worker entirely, decouples pusher and django and makes things a bit easier at the moment.

来源：https://stackoverflow.com/questions/50130851/django-channels-background-worker-instances

标签

python

django

pusher

channels