DEV: (cmds+) updates for streams idempotent production #2637

dwdougherty · 2026-01-13T21:09:07Z

No description provided.

github-actions · 2026-01-13T21:09:21Z

github-actions · 2026-01-13T21:09:24Z

LiorKogan · 2026-01-14T06:33:02Z

content/develop/data-types/streams/idempotency.md

+Configure idempotency settings for a stream using [`XCFGSET`]({{< relref "/commands/xcfgset" >}}):
+
+```
+XCFGSET mystream DURATION 300 MAXSIZE 1000


Parameter names were changed yesterday to IDMP-DURATION and IDMP-MAXSIZE.
Sorry for that.

LiorKogan · 2026-01-14T06:35:56Z

content/commands/xadd.md

+Enables idempotent message processing (at-most-once production) to prevent duplicate entries. Available since Redis 8.6.
+
+- `IDMPAUTO producer-id`: Automatically generates a unique idempotent ID (iid) for the specified producer-id. Redis tracks this iid to prevent duplicate messages from the same producer-id.
+- `IDMP producer-id idempotent-id`: Uses the specified idempotent-id for the given producer-id. If this producer-id/idempotent-id combination was already used, the command returns the idempotent-id of the existing entry instead of creating a duplicate.


the command returns the ID of the previously-added entry (like you mentioned below)

Not the idempotent-id.

LiorKogan · 2026-01-14T08:11:45Z

content/commands/xcfgset.md

+- **ERR no such key**: The stream does not exist
+- **ERR syntax error**: Invalid command syntax or missing required arguments
+- **ERR invalid duration**: Duration value is outside the valid range (1-86400)
+- **ERR invalid maxsize**: Maxsize value is outside the valid range (1-10,000)


Nitpick: "10,000" - different format from "64000"

LiorKogan · 2026-01-14T08:38:11Z

content/develop/data-types/streams/idempotency.md

+weight: 10
+---
+
+In Redis 8.6, streams support idempotent message processing (at-most-once production) to prevent duplicate entries when using at-least-once delivery patterns. This feature enables reliable message submission with automatic deduplication.


Idempotent message processing ensures that handling the same message multiple times produces the same system state as handling it once.

Beginning with Redis 8.6, streams support idempotent message processing (at-most-once production) to prevent duplicate entries when producers resend messages.

Producers may need to resend messages under two scenarios:

Producer-Redis network issues (i.e., disconnection and reconnection)

If a disconnection occurs after the producer executes XADD, but before it receives the reply, the producer has no way of knowing if that message was delivered to Redis.

The producer crashes and restarts

If the producer crashes after calling XADD but before receiving the reply and marking a message as “delivered”, after a restart, the producer has no way of knowing if that message was delivered to Redis.

In both cases, to guarantee that the message is added to the stream, the producer must call XADD again with the same message. Without idempotent message processing, a retry may result in a message being delivered twice. With idempotent message processing, producers can guarantee at-most-once production even under such scenarios.

LiorKogan · 2026-01-14T08:46:35Z

content/develop/data-types/streams/idempotency.md

+Use the [`XADD`]({{< relref "/commands/xadd" >}}) command with idempotency parameters, `IDMP` or `IDMPAUTO`:
+
+```
+XADD mystream IDMP producer1 msg1 * field value      # producer-provided iid


I'm not sure it is clear that producer1 is the pid and msg1 is the iid.

LiorKogan · 2026-01-14T08:51:41Z

content/develop/data-types/streams/idempotency.md

+XADD mystream IDMPAUTO producer1 * field value
+```
+
+- `pid`: Unique identifier for the message producer.


We should stress that

For both IDMP and IDMPAUTO:

Each producer app is required to use the same pid after it restarts.

For IDMP:

Each producer app is responsible for

Providing a unique iid for each entry (either globally, or just for each pid)

Reusing the same (pid, iid) when resending a message (even after it restarts)

LiorKogan · 2026-01-14T08:54:38Z

content/develop/data-types/streams/idempotency.md

+XADD mystream IDMP producer2 msg1 * field value    # Producer 2's tracking (independent)
+```
+
+Producers can use the same idempotent ID without conflicts.


as long as they use different pids.

LiorKogan · 2026-01-14T08:58:00Z

content/develop/data-types/streams/idempotency.md

+Idempotent IDs are removed when either condition is met:
+
+- Time-based: iids expire after the configured `DURATION`.
+- Capacity-based: Oldest iids are evicted when `MAXSIZE` is reached.


It is important to clarify that Redis never keeps more than maxsize iids per pid (in other words, MAXSIZE is "stronger" than DURATION).

LiorKogan · 2026-01-14T09:01:55Z

content/develop/data-types/streams/idempotency.md

+
+- `idmp-duration`: Current duration setting.
+- `idmp-maxsize`: Current maxsize setting.
+- `pids-tracked`: Number of active producers.


I'd avoid "active" as it is ambiguous.
I'd replace "active producers" with "the number of pids currently tracked in the stream".
(because an app may use the same pid for multiple producers, though it is wrong to do so).

LiorKogan · 2026-01-14T09:02:24Z

content/develop/data-types/streams/idempotency.md

+- `idmp-duration`: Current duration setting.
+- `idmp-maxsize`: Current maxsize setting.
+- `pids-tracked`: Number of active producers.
+- `iids-tracked`: Total idempotent IDs currently stored.


stored --> tracked.

LiorKogan · 2026-01-14T09:03:03Z

content/develop/data-types/streams/idempotency.md

+- `idmp-maxsize`: Current maxsize setting.
+- `pids-tracked`: Number of active producers.
+- `iids-tracked`: Total idempotent IDs currently stored.
+- `iids-added`: Lifetime count of idempotent messages added.


idempotent messages --> messages with idempotent ids.

LiorKogan · 2026-01-14T09:04:12Z

content/develop/data-types/streams/idempotency.md

+Use globally unique, persistent producer IDs:
+
+- Recommended: UUID v4 for global uniqueness.
+- Alternative: `hostname:process_id` or application-assigned IDs.


I don't think it is a good idea. The process_id will likely change after a restart.
Is it a ChatGPT suggestion?

LiorKogan · 2026-01-14T09:06:28Z

content/develop/data-types/streams/idempotency.md

+
+Use globally unique, persistent producer IDs:
+
+- Recommended: UUID v4 for global uniqueness.


UUID for a producer ID is usually overkill.
Apps should use shorter pids (if possible). This would save memory and increase performance.

LiorKogan · 2026-01-14T09:09:36Z

content/develop/data-types/streams/idempotency.md

+
+- RDB/AOF: All producer-idempotent ID pairs are saved.
+- Recovery: Tracking remains active after restart.
+- Configuration: `DURATION` and `MAXSIZE` settings persist.


IDMP-DURATION, IDMP-MAXSIZE

LiorKogan · 2026-01-14T09:11:17Z

content/develop/data-types/streams/idempotency.md

+- RDB/AOF: All producer-idempotent ID pairs are saved.
+- Recovery: Tracking remains active after restart.
+- Configuration: `DURATION` and `MAXSIZE` settings persist.
+- Important: Running `XCFGSET` clears all existing tracking data. Use without `DURATION` and `MAXSIZE` to clear data. 


I don't think so.
The requirement was: Calling XCFGSET key with different duration or maxsize values than the current, clears the IDMP map for the given key.
So if the values are the same - do nothing.
I'm not sure if you can call it without any values. And even if you can - maybe it uses the default values?

LiorKogan · 2026-01-14T09:15:08Z

content/commands/xadd.md

 XRANGE mystream - +
 {{% /redis-cli %}}

+### Idempotent message processing examples


I'm not sure if this is mentioned:

IDMPAUTO pid and IDMP pid iid can be specified only when the entry id is *.

andy-stark-redis

Some tiny things to check out, but otherwise language LGTM.

andy-stark-redis · 2026-01-14T09:33:16Z

content/develop/data-types/streams/idempotency.md

+You should know how long it may take your producer application to recover from a crash and start resending messages. 
+`DURATION` should be set accordingly.


Avoid PV in the otherwise AV sentence? (Not major)

Suggested change

You should know how long it may take your producer application to recover from a crash and start resending messages.

`DURATION` should be set accordingly.

You should know how long it may take your producer application to recover from a crash and start resending messages,

so you should set `DURATION` accordingly.

andy-stark-redis · 2026-01-14T09:35:10Z

content/develop/data-types/streams/idempotency.md

+XINFO STREAM mystream
+```
+
+Returns additional fields when idempotency is begin used:


Suggested change

Returns additional fields when idempotency is begin used:

Returns additional fields when idempotency is being used:

andy-stark-redis · 2026-01-14T09:36:56Z

content/develop/data-types/streams/idempotency.md

+- Memory: <1.5% additional memory usage.
+- Latency: Negligible impact on per-operation latency.
+
+Manual mode (IDMP) is slightly faster than automatic mode (IDMPAUTO) due to avoiding hash calculation.


Maybe a bit neater?

Suggested change

Manual mode (IDMP) is slightly faster than automatic mode (IDMPAUTO) due to avoiding hash calculation.

Manual mode (IDMP) is slightly faster than automatic mode (IDMPAUTO) since it avoids hash calculation.

andy-stark-redis · 2026-01-14T09:40:10Z

content/develop/data-types/streams/idempotency.md

+
+Here's an illustration of how message processing in Redis Streams works with and without idempotent production:
+
+{{< image filename="images/dev/stream/stream-idempotency.png" alt="Idempotent message processing in Redis Streams" >}}


This could probably be done as a Mermaid diagram (more AI-friendly) but it's not essential - maybe getting this published quickly is the top priority for now!

dwdougherty added 2 commits January 12, 2026 14:33

DOC-6151: (DEV) update XADD for idempotent production (commit 1)

199c193

Flesh out content

a23c468

dwdougherty requested review from a team and LiorKogan January 13, 2026 21:09

dwdougherty self-assigned this Jan 13, 2026

dwdougherty added the ros 8.6 label Jan 13, 2026

dwdougherty changed the title ~~Doc 6151~~ DOC-6151: updates for for idempotent production Jan 13, 2026

dwdougherty changed the title ~~DOC-6151: updates for for idempotent production~~ DEV: (cmds+) updates for streams idempotent production Jan 13, 2026

LiorKogan reviewed Jan 14, 2026

View reviewed changes

andy-stark-redis approved these changes Jan 14, 2026

View reviewed changes

andy-stark-redis reviewed Jan 14, 2026

View reviewed changes


		Use globally unique, persistent producer IDs:

		- Recommended: UUID v4 for global uniqueness.

		You should know how long it may take your producer application to recover from a crash and start resending messages.
		`DURATION` should be set accordingly.

	Returns additional fields when idempotency is begin used:
	Returns additional fields when idempotency is being used:

	Manual mode (IDMP) is slightly faster than automatic mode (IDMPAUTO) due to avoiding hash calculation.
	Manual mode (IDMP) is slightly faster than automatic mode (IDMPAUTO) since it avoids hash calculation.


		Here's an illustration of how message processing in Redis Streams works with and without idempotent production:

		{{< image filename="images/dev/stream/stream-idempotency.png" alt="Idempotent message processing in Redis Streams" >}}

DEV: (cmds+) updates for streams idempotent production #2637

Are you sure you want to change the base?

DEV: (cmds+) updates for streams idempotent production #2637

Uh oh!

Conversation

dwdougherty commented Jan 13, 2026

Uh oh!

github-actions bot commented Jan 13, 2026 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 13, 2026

Uh oh!

LiorKogan Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiorKogan Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiorKogan Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiorKogan Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andy-stark-redis left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions bot commented Jan 13, 2026 •

edited by atlassian bot

Loading

LiorKogan Jan 14, 2026 •

edited

Loading

LiorKogan Jan 14, 2026 •

edited

Loading

LiorKogan Jan 14, 2026 •

edited

Loading

LiorKogan Jan 14, 2026 •

edited

Loading