Bulk delete with Redisearch

Redis modules RediSearch
1

Hi there,

I am using Redisearch 1.6.13 with redis 6 and the lettusearch Java client.

The index is simple and looks like the following:

public static Schema VISITOR_SCHEMA = Schema.builder()
            .field(TagField.builder().name("projectId").build())
            .field(NumericField.builder().name("lastSeen").sortable(true).build())
            .build();

My purpose is to delete all documents that match a given projectId field from the index.

While looking at solutions I read that conditional deletes could be supported in the future but it seems that’s not already the case. As an alternative, I was thinking to create a Lua script that performs a search by tag with the FT.SEARCH command, then to iterate over the results to execute a bulk delete.

I started to create a Lua script as follows:

local projectId = KEYS[1]

local visitors = redis.call("FT.SEARCH", "visitor", "@projectId:{" .. projectId .. "} NOCONTENT")

Then, I tested it in debug mode from the redis-cli command to see what value is returned:

FT.SEARCH visitor @projectId:{7eBIKQDUSvFrOqx2vOHY} NOCONTENT
[0]

An empty Array seems returned. If I remove the NOCONTENT option, then I get the 3 documents I expect but with all fields. Is there a way to get only the document identifiers to iterate over for deletion? or is there a better approach?

Update:

Here is the script I currently use but it returns all fields before deleting:

local projectId = KEYS[1]

local visitors = redis.call("FT.SEARCH", "visitor", "@projectId:{" .. projectId .. "}")

local count = visitors[1]

for i = 2, (2 * count) + 1, 2 do
    redis.call("FT.DEL", "visitor", visitors[i], "DD")
end
2

@lpellegr please notice that you are missing a ‘,’ between the query and the ‘NOCONTENT’ argument. Maybe this is the issue?

3

Thanks for the suggestion. Unfortunately, it does return no results.

I updated the script query as follows:

local visitors = redis.call("FT.SEARCH", "visitor", "@projectId:{" .. projectId .. "}, NOCONTENT")

The script is executed in debug as follows:

$ redis-cli -h 127.0.0.1 -p 37493 -a "xxx" --ldb --eval delete-by-projectid.lua 4gi2Bmk8gvOI6jHz6oUP

Here is the debug output:

  • Stopped at 1, stop reason = step over
    → 1 local projectId = KEYS[1]
    lua debugger> s
  • Stopped at 3, stop reason = step over
    → 3 local visitors = redis.call(“FT.SEARCH”, “visitor”, “@projectId:{” … projectId … “}, NOCONTENT”)
    lua debugger> s
    FT.SEARCH visitor @projectId:{4gi2Bmk8gvOI6jHz6oUP}, NOCONTENT
    [0]

If I remove , NOCONTENT from the script query, I get the right entries but with all fields.

What is really weird is that if I execute the query manually with redis-cli (i.e. not within a script), then the query with , NOCONTENT returns the expected answer with the doc ID only:

FT.SEARCH visitor @projectId:{4gi2Bmk8gvOI6jHz6oUP}, NOCONTENT

  1. (integer) 22
    …

The problem seems to be with the Lua script and redis.call but I have no idea what I am doing wrong. Any ideas?

4

Try like this:
local visitors = redis.call("FT.SEARCH", "visitor", "@projectId:{" .. projectId .. "}", "NOCONTENT")

1 Like
5

That’s working. Thanks a lot!

1 Like