The path of one message#
A client runs SUBSCRIBE orders and its connection switches into a subscribed state.[1] Redis adds the connection to the subscriber list for orders (a dictionary from channel name to client list). PSUBSCRIBE pay* adds the connection to a separate list of glob patterns. That's all the state there is: subscriptions live on connections, in memory.[2]
PUBLISH orders "order#42"arrives on the publisher's connection.- Redis looks up the channel's subscriber list, then checks every pattern.[2]
- For each match it appends the message to that client's output buffer and replies to the publisher with the number of receivers.[2][3]
- Redis writes each buffer to its non-blocking socket as far as the kernel accepts. What doesn't fit stays in the output buffer, in Redis's memory.[3]
Each delivery is a RESP array: ["message", channel, payload], or ["pmessage", pattern, channel, payload] for pattern matches. A client subscribed to both a channel and a matching pattern receives the message twice.[1]
See it happen
At-most-once, no storage#
Pub/Sub has no relation to the key space, so messages never reach RDB snapshots, and PUBLISH is propagated to replicas but not written to the AOF.[1][2] Subscribers send no acknowledgements. If nobody is subscribed when PUBLISH runs, the message is discarded and the reply is 0. If a subscriber is restarting, it misses everything published in the meantime, and there's no offset to resume from. Each message is delivered at most once.[1]
The subscriptions aren't persisted either. When Redis restarts, every connection is gone along with its subscriptions. Until each client reconnects and runs SUBSCRIBE again, PUBLISH reaches nobody.[2]
Push delivery and the output buffer#
Pub/Sub is push-based: Redis writes to the subscriber whether or not it's ready.[4] Redis uses non-blocking sockets, so it never waits for a slow client. When the kernel's socket buffers are full (TCP flow control has closed the window), the remaining bytes wait in the client output buffer, in Redis's own memory.[3][5] A slow consumer becomes Redis's memory problem.
That's why Redis ships with a limit for Pub/Sub clients. The default:[3][6]
- Hard limit: the moment the buffer exceeds 32mb, the client is closed.
- Soft limit: if the buffer stays above 8mb continuously for 60 seconds, the client is closed. Dropping below resets the timer.[6]
When a client is closed, the messages still in its output buffer are lost.[1] Redis logs Client … scheduled to be closed ASAP for overcoming of output buffer limits.[7] A client that reconnects while still slow hits the limit again.
Without the limit#
Setting the limit to 0 0 0 removes it.[6] A stuck subscriber's buffer then grows without bound.
A slow reader and a dead network look the same#
If the network to a subscriber breaks silently, no FIN or RST arrives. Redis only sees that the TCP window stopped opening, which is exactly what a slow reader looks like, so the buffer grows until the limit closes the client.[5]
The subscriber only reads and never sends, so it may not notice for a long time: the connection is half-open. TCP keepalive (Redis defaults to tcp-keepalive 300 on its side) or an application-level PING health check is how clients detect it.[5][3][1]
See it happen
When to use something else#
Pub/Sub is the right tool when losing a message is acceptable; for stronger guarantees the Redis docs point to Streams.[1] The difference is whether the broker keeps the data and who sets the pace.
| Redis Pub/Sub | Redis Streams | Kafka | |
|---|---|---|---|
| Stores messages | No | Yes, as a key (persisted if RDB/AOF is on) | Yes, a replicated log on disk |
| Delivery | Push | Pull: XREAD / XREADGROUP, can BLOCK | Pull: consumers fetch |
| Guarantee | At-most-once | At-least-once with consumer groups and XACK | At-least-once by default |
| Consumer offline | Misses everything | Resumes from its group's last ID | Resumes from its committed offset |
| Slow consumer | Grows its output buffer in Redis until closed | Entries wait in the stream (until trimmed) | Lag grows (until retention deletes data) |
Sources: Pub/Sub[1], Streams[10], Kafka[11][12][13].
- Need to survive consumer restarts, or replay history? Use Redis Streams: an append-only log with consumer groups, acknowledgements (
XACK) and a pending entries list.[10] - Need durable, replayable messaging where each consumer sets its own pace? Use a pull-based log like Kafka: consumers fetch what they can handle and fall behind (lag) instead of filling a broker buffer.[11]
- On Redis Cluster 7+,
SSUBSCRIBE/SPUBLISH(sharded Pub/Sub) keep traffic on the shard that owns the channel's slot instead of broadcasting to every node.[1]