Redis Search and SortedSets, Eviction Policies

Hi,

I’m currently evaluating Redis with search module as a potential solution.

Scenario:

Say I have a set of customers, who have multiple accounts (1-10 per customer), and the accounts have multiple orders (10-1000 per account). On the front end of the app the customer is able to look at their various accounts, seeing paginated lists of orders. They’re also able to filter and sort the orders.

At the moment, this is all done via DB queries, but I’m looking to see if we can use Redis / search to improve performance.

When I was first looking at using Redis, the Orders seemed to fit quite nicely with using SortedSets - something like Orders:{CustomerId}:{AccountId} or Orders:{CustomerId}:{AccountId}:YYMM

That would let us store limited sets of data in Redis, and use the score for date sorting and limiting number of records returned. The vast majority of customer views are for recent data, so we can probably limit it to a few month’s worth for the cache, passing anything bigger back to the DB.

BUT because we have multiple possible queries, with different filtering criteria this seemed to get quite complicated, we’d have to start storing secondary lookups etc. So I started looking at Redisearch instead…

This would seem to give us multiple ways of sorting and filtering the data, direct from cache. Basically like non-clustered indexes in SQL.

However, I was then wondering about how the data should actually be stored for search. All the examples I’ve seen so far have used HashSets.

  • Could/would you use SortedSets with Redisearch?
  • If I used HashSets, how would I structure the data? Would it be an order per hashset?
  • If I used an order per hashset, how can I be sure that all the necessary data is in cache? (i.e. individual hashes could be evicted? With SortedSets it seems easier to keep track of what is/isn’t present)

Any other advice on how to approach this kind of problem?

Thanks!

Hey PhilSand,

Thanks for the detailed question! Here are some answers:

Could/would you use SortedSets with Redisearch?

RediSearch only indexes hashes today. There are some plans to support JSON in the future, but sorted sets are not planned.

If I used HashSets, how would I structure the data? Would it be an order per hashset?

Yes, one order per hash.

If I used an order per hashset, how can I be sure that all the necessary data is in cache? (i.e. individual hashes could be evicted? With SortedSets it seems easier to keep track of what is/isn’t present)

All of the data will always be stored in memory. For this use case, you’d disable eviction, so that nothing can be evicted.

Let me know if I can help further!

1 Like

Thanks Kyle! I managed to have a go with the Docker image of Redisearch yesterday and this all makes more sense now. Interesting point about disabling eviction - so you’d need to ensure you manually invalidate (or update) data.

That’s correct. You’d treat it just as you’d treat any other database.

Can you share which other structures are planned, for example, will LIST be supported for searching?

Hey local-admin,

The next planned data structures for indexing are JSON and Streams. Can you tell us more about your lists use case?

Kyle