Using Django Channels to subscribe to a Redis channel

[Jochen Breuer]

Hi!

I'm not entirely sure where to start and I hope you can help me. I'm using
SaltStack to execute tasks on minions. The results of those jobs (that's
the term used in Salt) are then published to a job cache. In this case its
a Redis server (Redis Pubsub). Now I would like to subscribe to a specific
Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.

Where to begin? Do I need to write a new protocol server or just a custom
channel? Even after reading the docs I'm still lacking overview. Perhaps
someone can push me into the right direction. Thanks!

Jochen

[Andrew Godwin]

Hi Jochen,

Your problem is that if you want to listen to the pubsub channel you will
need a dedicated process to do so, as you can't just poll something like
that. Given that restriction, you're going to have to write something like
a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.

You can send to channels from anywhere, so it's just a basic listener with
one line of channel send added. You're likely to want some sort of logic on
these messages, I suspect, so I would send onto a single custom channel and
then, now your messages are in the evented channel system, you can then
write a consumer for that channel and tie it into the routing and handle
any distribution/storage there.

This keeps the body of your logic inside the Django consumer code, and the
management command nice and simple - it just needs to send onto a custom
channel and not worry about what to do with that data, you can hook that up
later. If you have fields in the message you split out into different
dictionary keys, you can even use the Channels routing options on those
fields - so, for example, you could route everything with a certain task
name to one consumer, and everything else to a catch-all consumer.

Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm using
SaltStack to execute tasks on minions. The results of those jobs (that's
the term used in Salt) are then published to a job cache. In this case its
a Redis server (Redis Pubsub). Now I would like to subscribe to a specific
Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a custom
channel? Even after reading the docs I'm still lacking overview. Perhaps
someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/
msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Jochen Breuer]

Hi Andrew,

thanks for your very detailed answer! This is much simpler that
anticipated. You are absolutely right, a management command is the most
simple approach here. That was the missing piece in my picture. Thank you
very much!

Jochen

Post by Andrew Godwin
Hi Jochen,
Your problem is that if you want to listen to the pubsub channel you will
need a dedicated process to do so, as you can't just poll something like
that. Given that restriction, you're going to have to write something like
a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.
You can send to channels from anywhere, so it's just a basic listener with
one line of channel send added. You're likely to want some sort of logic on
these messages, I suspect, so I would send onto a single custom channel and
then, now your messages are in the evented channel system, you can then
write a consumer for that channel and tie it into the routing and handle
any distribution/storage there.
This keeps the body of your logic inside the Django consumer code, and the
management command nice and simple - it just needs to send onto a custom
channel and not worry about what to do with that data, you can hook that up
later. If you have fields in the message you split out into different
dictionary keys, you can even use the Channels routing options on those
fields - so, for example, you could route everything with a certain task
name to one consumer, and everything else to a catch-all consumer.
Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm
using SaltStack to execute tasks on minions. The results of those jobs
(that's the term used in Salt) are then published to a job cache. In this
case its a Redis server (Redis Pubsub). Now I would like to subscribe to a
specific Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a custom
channel? Even after reading the docs I'm still lacking overview. Perhaps
someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Michael]

Did you have luck with this. I've tried the same thing, but it seems like
if no one is consuming on the other side I get a
"channels.exceptions.ChannelFull" exception.

I'm trying to provide a real-time feed for an exchange. There might be zero
people listing or 1 million listening. The high level design should be the
same.

Post by Jochen Breuer
Hi Andrew,
thanks for your very detailed answer! This is much simpler that
anticipated. You are absolutely right, a management command is the most
simple approach here. That was the missing piece in my picture. Thank you
very much!
Jochen

Post by Andrew Godwin
Hi Jochen,
Your problem is that if you want to listen to the pubsub channel you will
need a dedicated process to do so, as you can't just poll something like
that. Given that restriction, you're going to have to write something like
a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.
You can send to channels from anywhere, so it's just a basic listener
with one line of channel send added. You're likely to want some sort of
logic on these messages, I suspect, so I would send onto a single custom
channel and then, now your messages are in the evented channel system, you
can then write a consumer for that channel and tie it into the routing and
handle any distribution/storage there.
This keeps the body of your logic inside the Django consumer code, and
the management command nice and simple - it just needs to send onto a
custom channel and not worry about what to do with that data, you can hook
that up later. If you have fields in the message you split out into
different dictionary keys, you can even use the Channels routing options on
those fields - so, for example, you could route everything with a certain
task name to one consumer, and everything else to a catch-all consumer.
Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm
using SaltStack to execute tasks on minions. The results of those jobs
(that's the term used in Salt) are then published to a job cache. In this
case its a Redis server (Redis Pubsub). Now I would like to subscribe to a
specific Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a
custom channel? Even after reading the docs I'm still lacking overview.
Perhaps someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[morlandi]

I posted a working snippet here

https://gist.github.com/morlandi/bb915db7acef0ee0e4cb070921208610

It's a Django management command which subscribes a Redis channel, and upon
receiving a new message sends it to django-channel.

You can skip the logging and redis connection boilerplate, and take a look
at the loop() and broadcast() method which are the relevant part.

Post by Michael
Did you have luck with this. I've tried the same thing, but it seems like
if no one is consuming on the other side I get a
"channels.exceptions.ChannelFull" exception.
I'm trying to provide a real-time feed for an exchange. There might be
zero people listing or 1 million listening. The high level design should be
the same.

Post by Jochen Breuer
Hi Andrew,
thanks for your very detailed answer! This is much simpler that
anticipated. You are absolutely right, a management command is the most
simple approach here. That was the missing piece in my picture. Thank you
very much!
Jochen

Post by Andrew Godwin
Hi Jochen,
Your problem is that if you want to listen to the pubsub channel you
will need a dedicated process to do so, as you can't just poll something
like that. Given that restriction, you're going to have to write something
like a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.
You can send to channels from anywhere, so it's just a basic listener
with one line of channel send added. You're likely to want some sort of
logic on these messages, I suspect, so I would send onto a single custom
channel and then, now your messages are in the evented channel system, you
can then write a consumer for that channel and tie it into the routing and
handle any distribution/storage there.
This keeps the body of your logic inside the Django consumer code, and
the management command nice and simple - it just needs to send onto a
custom channel and not worry about what to do with that data, you can hook
that up later. If you have fields in the message you split out into
different dictionary keys, you can even use the Channels routing options on
those fields - so, for example, you could route everything with a certain
task name to one consumer, and everything else to a catch-all consumer.
Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm
using SaltStack to execute tasks on minions. The results of those jobs
(that's the term used in Salt) are then published to a job cache. In this
case its a Redis server (Redis Pubsub). Now I would like to subscribe to a
specific Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a
custom channel? Even after reading the docs I'm still lacking overview.
Perhaps someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Nasir Sh]

Thanks Andrew this helps me as well. There is a missing piece I don't
understand yet. In my management command, then I probably will have to use `async_to_sync(channel_layer.group_send)`
to send the message to consumers (right?). The only problem is that
whenever I use async_to_sync it creates a new connection to redis every
single time. I now use channels_redis v2.3.1 and in this version I don't
get excessive connections but still it opens a new connection on every
send. I was wondering if that is intentional or not in a sync context and
if so, how can it be solved.

Thanks again for your great work

Post by Andrew Godwin
Hi Jochen,
Your problem is that if you want to listen to the pubsub channel you will
need a dedicated process to do so, as you can't just poll something like
that. Given that restriction, you're going to have to write something like
a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.
You can send to channels from anywhere, so it's just a basic listener with
one line of channel send added. You're likely to want some sort of logic on
these messages, I suspect, so I would send onto a single custom channel and
then, now your messages are in the evented channel system, you can then
write a consumer for that channel and tie it into the routing and handle
any distribution/storage there.
This keeps the body of your logic inside the Django consumer code, and the
management command nice and simple - it just needs to send onto a custom
channel and not worry about what to do with that data, you can hook that up
later. If you have fields in the message you split out into different
dictionary keys, you can even use the Channels routing options on those
fields - so, for example, you could route everything with a certain task
name to one consumer, and everything else to a catch-all consumer.
Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm
using SaltStack to execute tasks on minions. The results of those jobs
(that's the term used in Salt) are then published to a job cache. In this
case its a Redis server (Redis Pubsub). Now I would like to subscribe to a
specific Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a custom
channel? Even after reading the docs I'm still lacking overview. Perhaps
someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Yavin Aalto Arba]

try with self.channel_layer.group_send ?

Post by Nasir Sh
Thanks Andrew this helps me as well. There is a missing piece I don't
understand yet. In my management command, then I probably will have to use `async_to_sync(channel_layer.group_send)`
to send the message to consumers (right?). The only problem is that
whenever I use async_to_sync it creates a new connection to redis every
single time. I now use channels_redis v2.3.1 and in this version I don't
get excessive connections but still it opens a new connection on every
send. I was wondering if that is intentional or not in a sync context and
if so, how can it be solved.
Thanks again for your great work

Post by Andrew Godwin
Hi Jochen,
Your problem is that if you want to listen to the pubsub channel you will
need a dedicated process to do so, as you can't just poll something like
that. Given that restriction, you're going to have to write something like
a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.
You can send to channels from anywhere, so it's just a basic listener
with one line of channel send added. You're likely to want some sort of
logic on these messages, I suspect, so I would send onto a single custom
channel and then, now your messages are in the evented channel system, you
can then write a consumer for that channel and tie it into the routing and
handle any distribution/storage there.
This keeps the body of your logic inside the Django consumer code, and
the management command nice and simple - it just needs to send onto a
custom channel and not worry about what to do with that data, you can hook
that up later. If you have fields in the message you split out into
different dictionary keys, you can even use the Channels routing options on
those fields - so, for example, you could route everything with a certain
task name to one consumer, and everything else to a catch-all consumer.
Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm
using SaltStack to execute tasks on minions. The results of those jobs
(that's the term used in Salt) are then published to a job cache. In this
case its a Redis server (Redis Pubsub). Now I would like to subscribe to a
specific Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a
custom channel? Even after reading the docs I'm still lacking overview.
Perhaps someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/800a3278-ba4b-402d-b8ee-341b999879ba%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/800a3278-ba4b-402d-b8ee-341b999879ba%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Nasir Shadravan]

Thanks Yyavin, the problem is that in my management command I don't have
access to `self` to send it. Is it possible to get access to a consumer
from outside or how should my data producer relay the messages to channel
in an async way?

Post by Yavin Aalto Arba
try with self.channel_layer.group_send ?

Post by Nasir Sh
Thanks Andrew this helps me as well. There is a missing piece I don't
understand yet. In my management command, then I probably will have to use `async_to_sync(channel_layer.group_send)`
to send the message to consumers (right?). The only problem is that
whenever I use async_to_sync it creates a new connection to redis every
single time. I now use channels_redis v2.3.1 and in this version I don't
get excessive connections but still it opens a new connection on every
send. I was wondering if that is intentional or not in a sync context and
if so, how can it be solved.
Thanks again for your great work

Post by Andrew Godwin
Hi Jochen,
Your problem is that if you want to listen to the pubsub channel you
will need a dedicated process to do so, as you can't just poll something
like that. Given that restriction, you're going to have to write something
like a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.
You can send to channels from anywhere, so it's just a basic listener
with one line of channel send added. You're likely to want some sort of
logic on these messages, I suspect, so I would send onto a single custom
channel and then, now your messages are in the evented channel system, you
can then write a consumer for that channel and tie it into the routing and
handle any distribution/storage there.
This keeps the body of your logic inside the Django consumer code, and
the management command nice and simple - it just needs to send onto a
custom channel and not worry about what to do with that data, you can hook
that up later. If you have fields in the message you split out into
different dictionary keys, you can even use the Channels routing options on
those fields - so, for example, you could route everything with a certain
task name to one consumer, and everything else to a catch-all consumer.
Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm
using SaltStack to execute tasks on minions. The results of those jobs
(that's the term used in Salt) are then published to a job cache. In this
case its a Redis server (Redis Pubsub). Now I would like to subscribe to a
specific Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a
custom channel? Even after reading the docs I'm still lacking overview.
Perhaps someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/800a3278-ba4b-402d-b8ee-341b999879ba%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/800a3278-ba4b-402d-b8ee-341b999879ba%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CA%2B%2Be-ZVtt3kw9g-t2_D44aVvjRtBebbUTDbCdnKSDUWYBDRcBQ%40mail.gmail.com
<https://groups.google.com/d/msgid/django-users/CA%2B%2Be-ZVtt3kw9g-t2_D44aVvjRtBebbUTDbCdnKSDUWYBDRcBQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
Regards
Nasir Shadravan

[Nasir Sh]

Thanks Yyavin, the problem is that in my management command I don't have
access to `self` to send it. Is it possible to get access to a consumer
from outside or how should my data producer relay the messages to channel
in an async way?

Post by Yavin Aalto Arba
try with self.channel_layer.group_send ?

Post by Nasir Sh
Thanks Andrew this helps me as well. There is a missing piece I don't
understand yet. In my management command, then I probably will have to use `async_to_sync(channel_layer.group_send)`
to send the message to consumers (right?). The only problem is that
whenever I use async_to_sync it creates a new connection to redis every
single time. I now use channels_redis v2.3.1 and in this version I don't
get excessive connections but still it opens a new connection on every
send. I was wondering if that is intentional or not in a sync context and
if so, how can it be solved.
Thanks again for your great work

Post by Andrew Godwin
Hi Jochen,
Your problem is that if you want to listen to the pubsub channel you
will need a dedicated process to do so, as you can't just poll something
like that. Given that restriction, you're going to have to write something
like a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.
You can send to channels from anywhere, so it's just a basic listener
with one line of channel send added. You're likely to want some sort of
logic on these messages, I suspect, so I would send onto a single custom
channel and then, now your messages are in the evented channel system, you
can then write a consumer for that channel and tie it into the routing and
handle any distribution/storage there.
This keeps the body of your logic inside the Django consumer code, and
the management command nice and simple - it just needs to send onto a
custom channel and not worry about what to do with that data, you can hook
that up later. If you have fields in the message you split out into
different dictionary keys, you can even use the Channels routing options on
those fields - so, for example, you could route everything with a certain
task name to one consumer, and everything else to a catch-all consumer.
Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm
using SaltStack to execute tasks on minions. The results of those jobs
(that's the term used in Salt) are then published to a job cache. In this
case its a Redis server (Redis Pubsub). Now I would like to subscribe to a
specific Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a
custom channel? Even after reading the docs I'm still lacking overview.
Perhaps someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/800a3278-ba4b-402d-b8ee-341b999879ba%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/800a3278-ba4b-402d-b8ee-341b999879ba%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Yavin Aalto Arba]

You need to set up the consumer to perform this task.

cf. https://github.com/justdjango/justchat/blob/master/src/chat/consumers.py

this corresponds to the following video tutorial which might be useful to
you:

Post by Nasir Shadravan
Thanks Yyavin, the problem is that in my management command I don't have
access to `self` to send it. Is it possible to get access to a consumer
from outside or how should my data producer relay the messages to channel
in an async way?

Post by Yavin Aalto Arba
try with self.channel_layer.group_send ?

Post by Nasir Sh
Thanks Andrew this helps me as well. There is a missing piece I don't
understand yet. In my management command, then I probably will have to use `async_to_sync(channel_layer.group_send)`
to send the message to consumers (right?). The only problem is that
whenever I use async_to_sync it creates a new connection to redis every
single time. I now use channels_redis v2.3.1 and in this version I don't
get excessive connections but still it opens a new connection on every
send. I was wondering if that is intentional or not in a sync context and
if so, how can it be solved.
Thanks again for your great work

Post by Andrew Godwin
Hi Jochen,
Your problem is that if you want to listen to the pubsub channel you
will need a dedicated process to do so, as you can't just poll something
like that. Given that restriction, you're going to have to write something
like a management command that opens a connection to Redis and listens for
messages, and then whenever it gets one, sends the message onto a channel
in the Django channel system.
You can send to channels from anywhere, so it's just a basic listener
with one line of channel send added. You're likely to want some sort of
logic on these messages, I suspect, so I would send onto a single custom
channel and then, now your messages are in the evented channel system, you
can then write a consumer for that channel and tie it into the routing and
handle any distribution/storage there.
This keeps the body of your logic inside the Django consumer code, and
the management command nice and simple - it just needs to send onto a
custom channel and not worry about what to do with that data, you can hook
that up later. If you have fields in the message you split out into
different dictionary keys, you can even use the Channels routing options on
those fields - so, for example, you could route everything with a certain
task name to one consumer, and everything else to a catch-all consumer.
Andrew

Post by Jochen Breuer
Hi!
I'm not entirely sure where to start and I hope you can help me. I'm
using SaltStack to execute tasks on minions. The results of those jobs
(that's the term used in Salt) are then published to a job cache. In this
case its a Redis server (Redis Pubsub). Now I would like to subscribe to a
specific Redis channel, where the job results are published, with my Django
(Channels) application. Every time a job result is pushed, I'd also like to
push a message to a channel in Django.
Where to begin? Do I need to write a new protocol server or just a
custom channel? Even after reading the docs I'm still lacking overview.
Perhaps someone can push me into the right direction. Thanks!
Jochen
--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/c933a5b5-7b72-4d7d-985a-a311c0f16b26%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/800a3278-ba4b-402d-b8ee-341b999879ba%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/800a3278-ba4b-402d-b8ee-341b999879ba%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/730e9a11-4226-4c7e-8252-e33518634418%40googlegroups.com
<https://groups.google.com/d/msgid/django-users/730e9a11-4226-4c7e-8252-e33518634418%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.