diff options
Diffstat (limited to 'iredis/data/commands/xreadgroup.md')
| -rw-r--r-- | iredis/data/commands/xreadgroup.md | 170 |
1 files changed, 85 insertions, 85 deletions
diff --git a/iredis/data/commands/xreadgroup.md b/iredis/data/commands/xreadgroup.md index f9201e4..4b516e5 100644 --- a/iredis/data/commands/xreadgroup.md +++ b/iredis/data/commands/xreadgroup.md @@ -1,103 +1,74 @@ -The `XREADGROUP` command is a special version of the `XREAD` command with -support for consumer groups. Probably you will have to understand the `XREAD` -command before reading this page will makes sense. +The `XREADGROUP` command is a special version of the `XREAD` command +with support for consumer groups. Probably you will have to understand the +`XREAD` command before reading this page will makes sense. Moreover, if you are new to streams, we recommend to read our -[introduction to Redis Streams](/topics/streams-intro). Make sure to understand -the concept of consumer group in the introduction so that following how this -command works will be simpler. +[introduction to Redis Streams](/topics/streams-intro). +Make sure to understand the concept of consumer group in the introduction +so that following how this command works will be simpler. ## Consumer groups in 30 seconds -The difference between this command and the vanilla `XREAD` is that this one -supports consumer groups. - -Without consumer groups, just using `XREAD`, all the clients are served with all -the entries arriving in a stream. Instead using consumer groups with -`XREADGROUP`, it is possible to create groups of clients that consume different -parts of the messages arriving in a given stream. If, for instance, the stream -gets the new entries A, B, and C and there are two consumers reading via a -consumer group, one client will get, for instance, the messages A and C, and the -other the message B, and so forth. - -Within a consumer group, a given consumer (that is, just a client consuming -messages from the stream), has to identify with an unique _consumer name_. Which -is just a string. - -One of the guarantees of consumer groups is that a given consumer can only see -the history of messages that were delivered to it, so a message has just a -single owner. However there is a special feature called _message claiming_ that -allows other consumers to claim messages in case there is a non recoverable -failure of some consumer. In order to implement such semantics, consumer groups -require explicit acknowledgment of the messages successfully processed by the -consumer, via the `XACK` command. This is needed because the stream will track, -for each consumer group, who is processing what message. +The difference between this command and the vanilla `XREAD` is that this +one supports consumer groups. + +Without consumer groups, just using `XREAD`, all the clients are served with all the entries arriving in a stream. Instead using consumer groups with `XREADGROUP`, it is possible to create groups of clients that consume different parts of the messages arriving in a given stream. If, for instance, the stream gets the new entries A, B, and C and there are two consumers reading via a consumer group, one client will get, for instance, the messages A and C, and the other the message B, and so forth. + +Within a consumer group, a given consumer (that is, just a client consuming messages from the stream), has to identify with a unique *consumer name*. Which is just a string. + +One of the guarantees of consumer groups is that a given consumer can only see the history of messages that were delivered to it, so a message has just a single owner. However there is a special feature called *message claiming* that allows other consumers to claim messages in case there is a non recoverable failure of some consumer. In order to implement such semantics, consumer groups require explicit acknowledgment of the messages successfully processed by the consumer, via the `XACK` command. This is needed because the stream will track, for each consumer group, who is processing what message. This is how to understand if you want to use a consumer group or not: -1. If you have a stream and multiple clients, and you want all the clients to - get all the messages, you do not need a consumer group. -2. If you have a stream and multiple clients, and you want the stream to be - _partitioned_ or _sharded_ across your clients, so that each client will get - a sub set of the messages arriving in a stream, you need a consumer group. +1. If you have a stream and multiple clients, and you want all the clients to get all the messages, you do not need a consumer group. +2. If you have a stream and multiple clients, and you want the stream to be *partitioned* or *sharded* across your clients, so that each client will get a sub set of the messages arriving in a stream, you need a consumer group. ## Differences between XREAD and XREADGROUP -From the point of view of the syntax, the commands are almost the same, however -`XREADGROUP` _requires_ a special and mandatory option: +From the point of view of the syntax, the commands are almost the same, +however `XREADGROUP` *requires* a special and mandatory option: GROUP <group-name> <consumer-name> The group name is just the name of a consumer group associated to the stream. -The group is created using the `XGROUP` command. The consumer name is the string -that is used by the client to identify itself inside the group. The consumer is -auto created inside the consumer group the first time it is saw. Different -clients should select a different consumer name. +The group is created using the `XGROUP` command. The consumer name is the +string that is used by the client to identify itself inside the group. +The consumer is auto created inside the consumer group the first time it +is saw. Different clients should select a different consumer name. -When you read with `XREADGROUP`, the server will _remember_ that a given message -was delivered to you: the message will be stored inside the consumer group in -what is called a Pending Entries List (PEL), that is a list of message IDs -delivered but not yet acknowledged. +When you read with `XREADGROUP`, the server will *remember* that a given +message was delivered to you: the message will be stored inside the +consumer group in what is called a Pending Entries List (PEL), that is +a list of message IDs delivered but not yet acknowledged. -The client will have to acknowledge the message processing using `XACK` in order -for the pending entry to be removed from the PEL. The PEL can be inspected using -the `XPENDING` command. +The client will have to acknowledge the message processing using `XACK` +in order for the pending entry to be removed from the PEL. The PEL +can be inspected using the `XPENDING` command. The `NOACK` subcommand can be used to avoid adding the message to the PEL in -cases where reliability is not a requirement and the occasional message loss is -acceptable. This is equivalent to acknowledging the message when it is read. +cases where reliability is not a requirement and the occasional message loss +is acceptable. This is equivalent to acknowledging the message when it is read. -The ID to specify in the **STREAMS** option when using `XREADGROUP` can be one -of the following two: +The ID to specify in the **STREAMS** option when using `XREADGROUP` can +be one of the following two: -- The special `>` ID, which means that the consumer want to receive only - messages that were _never delivered to any other consumer_. It just means, - give me new messages. -- Any other ID, that is, 0 or any other valid ID or incomplete ID (just the - millisecond time part), will have the effect of returning entries that are - pending for the consumer sending the command with IDs greater than the one - provided. So basically if the ID is not `>`, then the command will just let - the client access its pending entries: messages delivered to it, but not yet - acknowledged. Note that in this case, both `BLOCK` and `NOACK` are ignored. +* The special `>` ID, which means that the consumer want to receive only messages that were *never delivered to any other consumer*. It just means, give me new messages. +* Any other ID, that is, 0 or any other valid ID or incomplete ID (just the millisecond time part), will have the effect of returning entries that are pending for the consumer sending the command with IDs greater than the one provided. So basically if the ID is not `>`, then the command will just let the client access its pending entries: messages delivered to it, but not yet acknowledged. Note that in this case, both `BLOCK` and `NOACK` are ignored. -Like `XREAD` the `XREADGROUP` command can be used in a blocking way. There are -no differences in this regard. +Like `XREAD` the `XREADGROUP` command can be used in a blocking way. There +are no differences in this regard. ## What happens when a message is delivered to a consumer? Two things: -1. If the message was never delivered to anyone, that is, if we are talking - about a new message, then a PEL (Pending Entries List) is created. -2. If instead the message was already delivered to this consumer, and it is just - re-fetching the same message again, then the _last delivery counter_ is - updated to the current time, and the _number of deliveries_ is incremented by - one. You can access those message properties using the `XPENDING` command. +1. If the message was never delivered to anyone, that is, if we are talking about a new message, then a PEL (Pending Entries List) is created. +2. If instead the message was already delivered to this consumer, and it is just re-fetching the same message again, then the *last delivery counter* is updated to the current time, and the *number of deliveries* is incremented by one. You can access those message properties using the `XPENDING` command. ## Usage example -Normally you use the command like that in order to get new messages and process -them. In pseudo-code: +Normally you use the command like that in order to get new messages and +process them. In pseudo-code: ``` WHILE true @@ -119,29 +90,58 @@ END ``` In this way the example consumer code will fetch only new messages, process -them, and acknowledge them via `XACK`. However the example code above is not -complete, because it does not handle recovering after a crash. What will happen -if we crash in the middle of processing messages, is that our messages will -remain in the pending entries list, so we can access our history by giving -`XREADGROUP` initially an ID of 0, and performing the same loop. Once providing -an ID of 0 the reply is an empty set of messages, we know that we processed and -acknowledged all the pending messages: we can start to use `>` as ID, in order -to get the new messages and rejoin the consumers that are processing new things. +them, and acknowledge them via `XACK`. However the example code above is +not complete, because it does not handle recovering after a crash. What +will happen if we crash in the middle of processing messages, is that our +messages will remain in the pending entries list, so we can access our +history by giving `XREADGROUP` initially an ID of 0, and performing the same +loop. Once providing an ID of 0 the reply is an empty set of messages, we +know that we processed and acknowledged all the pending messages: we +can start to use `>` as ID, in order to get the new messages and rejoin the +consumers that are processing new things. To see how the command actually replies, please check the `XREAD` command page. +## What happens when a pending message is deleted? + +Entries may be deleted from the stream due to trimming or explicit calls to `XDEL` at any time. +By design, Redis doesn't prevent the deletion of entries that are present in the stream's PELs. +When this happens, the PELs retain the deleted entries' IDs, but the actual entry payload is no longer available. +Therefore, when reading such PEL entries, Redis will return a null value in place of their respective data. + +Example: + +``` +> XADD mystream 1 myfield mydata +"1-0" +> XGROUP CREATE mystream mygroup 0 +OK +> XREADGROUP GROUP mygroup myconsumer STREAMS STREAMS mystream > +1) 1) "mystream" + 2) 1) 1) "1-0" + 2) 1) "myfield" + 2) "mydata" +> XDEL mystream 1-0 +(integer) 1 +> XREADGROUP GROUP mygroup myconsumer STREAMS STREAMS mystream 0 +1) 1) "mystream" + 2) 1) 1) "1-0" + 2) (nil) +``` + @return @array-reply, specifically: -The command returns an array of results: each element of the returned array is -an array composed of a two element containing the key name and the entries -reported for that key. The entries reported are full stream entries, having IDs -and the list of all the fields and values. Field and values are guaranteed to be -reported in the same order they were added by `XADD`. +The command returns an array of results: each element of the returned +array is an array composed of a two element containing the key name and +the entries reported for that key. The entries reported are full stream +entries, having IDs and the list of all the fields and values. Field and +values are guaranteed to be reported in the same order they were added +by `XADD`. When **BLOCK** is used, on timeout a null reply is returned. Reading the [Redis Streams introduction](/topics/streams-intro) is highly -suggested in order to understand more about the streams overall behavior and -semantics. +suggested in order to understand more about the streams overall behavior +and semantics. |