Structuring a stream for ordering of consumer groups

Redis commands & data structures
1

Hello, I’m working on a new use-case for Redis streams to orchestrate a 3 step process, and I’m having some trouble understanding how I could structure my data in the stream that would allow for ordering of the consumer groups. In that for each run I want to run a consumer group on a stream for step 1 that outputs a document, and when step 1 is completed successfully I want to run step 2 that reads the document and outputs another document, that would then be read by step 3 and complete the process.

Is there any advice on how I could structure my stream to allow ordering of these consumer groups? Would I need another data structure to do the locking until the proceeding step is completed?

Thanks for the help.

2

One way to structure this is to create a dedicated work queue per step. That is you need to create as many streams as you have steps, and consumer groups for each as well.

3

To add to what Julien said,

Here is a demo example of how you can choreograph a saga using Redis Streams - GitHub - redis-field-engineering/demo-microservices-saga

There is an explanation of microservice sagas here - https://thenewstack.io/how-redis-simplifies-microservices-design-patterns/