RedisJSON and RediSEARCH high offset on LIMIT with SORTBY much slower then low OFFSET

Redis modules
,
1

Hi to everyone,

I’m experiencing a query execution time dropdown when i request results with high offset.

If i have a ~600k documents where each document has around 9 keys and each key has a value that is a string and with average size of 10 and there is no further object nesting.

I created the index like following:
FT.CREATE name_index ON JSON SCHEMA $.name AS name TAG SORTABLE

The following query is executed really fast (40ms):
FT.SEARCH name_index * SORTBY name ASC

But when i try to fetch near the end, it’s being executed in 3.5 seconds
FT.SEARCH name_index * LIMIT 599990 10 SORTBY name ASC TIMEOUT 5000

When i’m using the profiler in RedisInsight, it says that sorting takes around 200ms, and pager/limiter takes around 90% of the execution.

Any advice how to make it a bit more performant if even possible?

Thanks in advance.

2

It seems like your query is trying to scan the all database and sort it all, is that what you meant?

Which version of RediSearch/Redis-Stack are you using?
Please notice the first milestone of RediSearch 2.8 includes major improvements in the SortBy

3

Yeah, at least thats what it looks like.

I’m using the latest Docker image: redis/redis-stack:latest, but the it says locally that the image was created 3 weeks ago. So i will try the one pushed 2 days ago.

EDIT:

I assume the edge tag is the one with latest update, as the OPTIMIZE keyword works in front of SORTBY.

But i see no difference at all.

I guess i fall under the no optimization mention.