In my microservice, I am consuming kafka events for millions of users and when a user connect to my microservice, I need to provide the user specific events to connected user.
These events has time-to-live for 10 minutes and event structure is like this:
{
"id":"63cd45d4-c6c4-462a-a668-efe410ed619e",
"event":"propChn",
"refId":"01HCFRNFXH8VR90MAE21M0E7NQ"
}
I am considering Redis for this use-case, though I am new to Redis, I found that Sorted Set score by Timestamp would be better for my use case and I am considering to create Sorted set per user
ZADD user:1 <timestamp> <event1> <event2>
ZADD user:1 <timestamp> <event3>
As I have millions of users, I also need to make sure to cleanup events which are older than 10 minutes form each user’s sorted set. Sorted Set Time-to-live feature work on complete Set and not the events stored in the Set.
Would there be any other efficient way to do this? Or is there any better way to cleanup events older than 10 minutes from all user’s (Sorted)Sets?