Anything to do to get RediSearch to search across clusters?

Redis modules RediSearch
1

I’m the author of a Redis client in Crystal that supports both clustering and various Redis Stack modules, including RediSearch, and I’m wondering if there’s anything that needs to be done for RediSearch to search across clusters or if the RediSearch module on clusters will handle this automatically.

My confusion about this stems from the fact that RSCoordinator hasn’t had any meaningful updates in 9 months, and this issue comment says it’s been rolled into RediSearch. But I can’t find any documentation on it and the FT.CREATE documentation still mentions it:

Screenshot from the FT.CREATE documentation page. Relevant excerpt reads: When having several indices in a clustered database, you need to make sure the documents you want to index reside on the same shard as the index. You can achieve this by having your documents tagged by the index name. When Running RediSearch in a clustered database, there is the ability to span the index across shards with RSCoordinator. In this case the above does not apply.
Screenshot from the FT.CREATE documentation page. Relevant excerpt reads: When having several indices in a clustered database, you need to make sure the documents you want to index reside on the same shard as the index. You can achieve this by having your documents tagged by the index name. When Running RediSearch in a clustered database, there is the ability to span the index across shards with RSCoordinator. In this case the above does not apply.1730×576 154 KB

Mainly, I want to know if my Redis client needs to accommodate RSCoordinator on clusters and, if so, what commands (or updates to existing commands) I need to add for it.

2

Can anyone shed any light on this? It’s been a year since RSCoordinator has had any commits and that was just to disable nightly CI.

I’m trying to understand how search works across shards. Testing on a local cluster seems to indicate that it isn’t an out-of-the-box thing because from what I could tell results were only coming back from one shard. Redis Enterprise DBs provisioned with sharding for RediSearch on app.redislabs.com seem to support this, though. I haven’t confirmed it because that’s an expensive test — I’m writing an OSS client, so a $640/mo database isn’t in my budget.

Where can I find more information about this?

3

Please notice the RSCoordinator project was deprecated.

It was merged long ago into the RediSearch project see: RediSearch/coord at master · RediSearch/RediSearch · GitHub

4

Perfect, does that mean distributed search works out of the box? Or is there anything I need to do to enable it? I unfortunately couldn’t find any docs about that. I’m have no doubt they exist, but last time I looked (around the time I originally posted this thread) I couldn’t find anything despite significant effort googling and looking through the RediSearch docs and GitHub issues. That’s where I found the random comment I linked in the OP about it being rolled into RediSearch itself, but it provided no further information or links to documentation about how it works, either.

5

any news about that? RediSearch works out of the box? or i need to enable it somehow?

6

To build it please see docs on building

make build COORD=oss will build OSS RediSearch Coordinator.

To run it please see docs on creating and running a cluster

In order to load the built RediSearch module, edit the create-cluster script:
ADDITIONAL_OPTIONS="--loadmodule <path to the built module-oss.so>"

7

I’m not asking about building RediSearch or running a cluster. My question is about RediSearch functionality while already running in a cluster.

I’m asking if I run FT.SEARCH on one node in my cluster, will RediSearch return results from other nodes in the same cluster or only that node? If RSCoordinator is part of RediSearch, then the part that the documentation about searching keys on one node sounds incorrect, but I don’t know what to trust.

My own experimentation only showed it returning results from one node, but I don’t know if that’s due to something in my own cluster configuration or a RediSearch setting. And since the documentation is outdated (still mentioning RSCoordinator as a separate project), I have no idea what to trust here.

8

When running with RSCoordinator you should get results from all the shards.

If you’re only getting results from a single shard it means that you’re running without RSCoordinator, what we call RediSearch light

9

@gkorland How do we run it with RSCoordinator? Is there a CONFIG value we need to set, maybe?

10

@gkorland Wait, is RSCoordinator still its own module? Because the following from further up the thread sounded like it’s just a part of the RediSearch module:

But this readme seems to indicate it’s still a separate module.

Adding to the confusion, the redis-stack-server container images (which seem to be Redis, Inc’s prescribed method of deployment) don’t include a .so file for RSCoordinator at all.

$ docker run --rm -it redis/redis-stack-server bash
root@af59cb2824dc:/# cd /opt/redis-stack/lib/
root@af59cb2824dc:/opt/redis-stack/lib# ls -l
total 33028
-rwxr-xr-x 1 nobody 123   643072 Feb  5 09:36 redisbloom.so
-rwxr-xr-x 1 nobody 123  4047432 Feb  5 09:36 redisearch.so
-rwxr-xr-x 1 nobody 123 23310256 Feb  5 09:36 redisgraph.so
-rwxr-xr-x 1 nobody 123  2225728 Feb  5 09:36 redistimeseries.so
-rwxr-xr-x 1 nobody 123  3586248 Feb  5 09:36 rejson.so

Is there documentation for how to use RSCoordinator in a redis-stack-server cluster?

11

As I wrote the coordinator is no longer a stand alone project but you can still build search or without it.

The redis-stack-server distribution (docker or binary) is not including rscoodinator.
We only include rscoodinator by default in the our cloud and enterprise distribution.

Omer shared with you Anything to do to get RediSearch to search across clusters? - #6 by oshadmi how to build Search from source with rscoodinator.