FT.AGGREGATE FT.ADD with tag fields

Redis modules RediSearch
1

Hi,
I’ve got a couple of questions.

  1. Let’s say I have an index with a single tag field (separator ‘;’) and I created a document with id: “test” with tags “tag1;tag2;tag3;tag4”. Can I add/remove the specific tag from tag field without knowing about other tags? For example, I want to remove “tag2” from the document mentioned before so then the value of tag field would look like this “tag1;tag3;tag4”.

  2. Let’s say I want to build dine table reservation system where every table has its own timeline. I want to use redis only for future reservations and for checking availability for a table at a specific time. Any reservation would be stored in separate, more durable database to save memory. So I want only check availability to of specific table and use transactions to prevent parallel reservation requests. I was thinking about tag field which would store timestamps of reservations and then I could group available tables by for example restaurant_id with FT.AGGREGATE. Le’ts say that the user wants reservation for 5:00 PM and the duration of the reservation is 1 hour. Additionally let’s assume that the user can make reservations for 5:00 PM, 5:15 PM, 5:30 PM, etc. so minimum 15min. Then query would look like this “-@tags:{timestamp of 5:00 PM} -@tags:{timestamp of 5:15 PM} -@tags:{timestamp of 5:30 PM} -@tags:{timestamp of 5:45 PM}”. That is my approach. Is it possible to model it in a different way (maybe more performant and without FT.AGGREGATE)"

  3. Referring to the previous question, is it good approach of using FT.AGGREGATE considering later scalability? Query in the previous question is much simpler than the final one that would work for my domain problem.

  4. Redisearch have ToList reducer for FT.AGGREGATE command. I want to group tables by restaurant_id and then get some fields from tables documents. Can I somehow retrieve more than one field with ToList? I tried to use couple of ToList reducers where each one retrieves a different field but I’m not sure that are they all in same order.
    For example:

    1. “restaurant_id”
    2. “c0124d5b-c2dd-11ea-817b-bcee7b9d9a5d”
    3. “tables”
      1. “testTable3”
      2. “testTable2”
      3. “testTable”
    5. “table_reservations”
      1. “t1;t2”
      2. “t1;t2;t3;t4”
      3. “t1;t2;t3;t4;t5;t6”

Is table “testTable3” corresponding to table_reservations “t1;t2” or is it just coincidence that they match?

  1. Is it possible to add a new field to index and then remove it? The dropped field from the index in all documents can be deleted in my case. Is it possible to approach this in a different way if dropping field from the index isn’t possible?

Thank you in advance for your answers.
Regards

2

Hi KiVo-

I’ll try and address your questions:

  1. No, there is on direct way of removing a tag from a field. All updates are done through FT.ADD (name is important) which can replace whole fields (or documents) only. Depending upon if you’re using RediSearch Enterprise or single node, you may be able to effectively do this with a one-line Lua script (get the values of the tag, remove as needed, update the whole field). Enterprise allows clustering, which complicates matters a bit, but there is still a solution to be had.

  2. I’m not sure I’d use FT.AGGREGATE and tags for that. The aggregation pipeline is needlessly complicated. I think I might use a document for each reservation and a start/end time as a number. You can normalize the time of day as a decimal number (e.g. 5pm is 17 and 5:45 is 17.75]. When you want to find if a table is reserved for a time period, do something like "@start:[17 16]|@end:[17 16]" this will catch any reservations starting or end during that period. If nothing is reserved, then it’s open. I will say there are a ton of ways to express this problem and the devil is in the detail.

  3. (and 4) tolist (and tags in general) have very limited ability to be worked with in the rest of the aggregation pipeline.

  4. It sounds like you are talking about altering the schema. In that case, no, you cannot remove schema fields. On an individual document basis you can remove fields (and a schema with a field that contains no data is pretty insignificant) but I’m not sure what you’re trying to do were.

-K

3

Thanks for your response.

I want to do a search like website OpenTable.com. They allow filter restaurants by many factors and the website shows appropriate free tables for each restaurant for specified time and side times. Like I said before I want only check availability of tables let’s say 30 min before and after the specified time. This is why I want group tables by restaurant and get 20 restaurants and available tables for each restaurant.

4

Gotcha. You could still do the aggregation then. The tags seem like an odd fit to me though. You may also want to consider a field for each time slot. Fields are pretty cheap.

5

What do you mean by fields? Do you suggest adding for each table field for each hour (8:00 AM. 8:15 AM etc.) and store there tags or what? I go a little confused. I need to have some sort of timetable for each day and each time slot. I honestly don’t know how would that work without tags. The main problem here is to find tables available of side times and specify which table is for side time and which is for the exact specified time of reservation. BTW, sorry for bothering you that much.

6

OK. When thinking this over, I had a few constraints in this solution.

  1. I wanted to stay away from finding tables using aggregations. This adds a lot of complexity and it can be accomplished without.
  2. Negative clauses are not super efficient, especially if you have all or most negative clauses.
  3. Balancing tag size vs field size. If you have a huge amount of bigger tags, that’s going to be recorded in Redis in a non-optimal way. If you have fields that are mostly unused and the field value is smaller, this will be smaller overall.
  4. You should only have to record the reservations, not empty dining tables

Here is what I came up with which I think is decent:

Schema:
FT.CREATE idx SCHEMA location TEXT NOSTEM h00 TEXT NOSTEM h01 TEXT NOSTEM h02 TEXT NOSTEM h03 TEXT NOSTEM h04 TEXT NOSTEM h05 TEXT NOSTEM h06 TEXT NOSTEM h07 TEXT NOSTEM h08 TEXT NOSTEM h09 TEXT NOSTEM h10 TEXT NOSTEM h11 TEXT NOSTEM h12 TEXT NOSTEM h13 TEXT NOSTEM h14 TEXT NOSTEM h15 TEXT NOSTEM h16 TEXT NOSTEM h17 TEXT NOSTEM h18 TEXT NOSTEM h19 TEXT NOSTEM h20 TEXT NOSTEM h21 TEXT NOSTEM h22 TEXT NOSTEM h23 TEXT NOSTEM

It may look complicated, but really it’s just a field for each time slot. This is simplified so you have hour long slots, but the same concept applies, just add more field (consider MAXTEXTFIELDS here too). I’m using NOSTEM as we’re effectively stripping out TEXT down for this use-case - you could also do more optimization here at the index level. I wanted to avoid TAG as you should only ever store a single value in each field.

Each document name is deterministic and knowable through some other method (mentioned later). It encodes the location and table info.

Example - we have three restaurants ABC, XYZ and 123

Let’s do a reservation for 8-10am (hour slots of 8-9 and 9-10) at location “ABC”:

FT.ADD idx ABCtable1 1 FIELDS location ABC h08 1 h09 1

Another for ABC from 6-8am at “ABC”:

FT.ADD idx ABCtable3 1 FIELDS location ABC h06 1 h07 1

At another location, “XYZ” from 5am to 7am:

FT.ADD idx XYZtable1 1 FIELDS location XYZ h05 1 h06 1

Let’s find out if there is a table at any restaurant from 10-11am:

> FT.SEARCH idx '@h10:"1"' NOCONTENT
1) (integer) 0

So - it’s open. Note: NOCONTENT is used here as the actual document is not super useful since the doc names encode metadata.

How about from 10am-Noon:

> FT.SEARCH idx '@h10|h11:"1"' NOCONTENT
1) (integer) 0

Still open.

How about from 9am to 11am:

> FT.SEARCH idx '@h09|h10:"1"' NOCONTENT
1) (integer) 1
2) "ABCtable1"

This is telling me that there is a reservation at ABC, but that XYZ and 123 don’t have reservations.

What about just location ABC or XYZ from 9am to 11am:

> FT.SEARCH idx '(@location:"ABC" | @location:"XYZ") @h09|h10:"1"' NOCONTENT
1) (integer) 1
2) "ABCtable1"

ABC has one reservation, but XYZ is open.

6-7am, location XYZ or ABC:

> FT.SEARCH idx '(@location:"ABC" | @location:"XYZ") @h06:"1"' NOCONTENT
1) (integer) 2
2) "XYZtable1"
3) "ABCtable3"

With regards to telling you what tables you have at locations, I’m not sure there is an efficient way of doing this in RediSearch alone (it gets into relational territory). If I were doing it with Redis alone, I would look at storing table information in plain old sets then just intersecting these sets with the results from the RediSearch query either with a Lua, Gears or your application logic since it will be comparatively small.

This method creates minimal documents without having hard to reason or execute queries.

7

Thank you so much for your time. I like this solution.
One more question to be completely clear:
I want to divide each timeslot into days so would be ok to do smth like this?

FT.ADD idx ABCtabl1 1 FIELDS location ABC h08 1-15072020 h10 1-15072020

and later another reservation the next day

FT.ADD idx ABCtabl1 1 FIELDS location ABC h08 “1-15072020 1-16072020” h10 “1-15072020 1-16072020”

and then search by date and time slot

FT.SEARCH idx 'h08:“1-15072020” ’ NOCONTENT

“15072020” is a date without any separator
Would it be efficient?

8

Ah - no. I wouldn’t do it that way. Seems like it would be a problem of removing previous days.

I would make an index for each day. Drop old indexes when not needed. You’ll have to namespace your doc names as well.

For July 15:

FT.ADD idx-200715 ABCtable1-200715 1 FIELDS location ABC h08 1 h09 1

At the end of the day, you would drop the entire index:

FT.DROP idx-200715

For July 16:

FT.ADD idx-200716 ABCtable1-200716 1 FIELDS location ABC h08 1 h09 1

Only downside here is that you would need do a bit of a special dance for reservations that pass midnight, but it’s not the worst thing.

(one thing that did occur to me is that you could not even use RediSearch at all and use GETBIT / SETBIT / BITOP to do all of this if you wanted to go real geeky :wink: )