How to get a quantile value from a reduce value ?
According documentation “each reducer works independently on each result and writes its final output once”, but how to do it in a single query ?
This query failed due to reducers working independently:
FT.AGGREGATE myIndex "*"
APPLY "@timestamp - (@timestamp % 3600)" AS hour
GROUPBY 1 @hour
REDUCE COUNT_DISTINCT 1 @user_id AS num_users
REDUCE QUANTILE 2 @num_users 50 AS median
> FT.AGGREGATE myIndex "*" APPLY "@timestamp - (@timestamp % 3600)" AS hour GROUPBY 1 @hour REDUCE COUNT_DISTINCT 1 @userid AS num_users GROUPBY 1 @num_users REDUCE QUANTILE 2 num_users 0.5 AS median
What’s going on here? The original query error message gives you a hint: Property num_users not present in document or pipeline the num_users property doesn’t make it into the pipeline until after the end of the GROUPBY. @meirsh, correct me if I’m wrong here.
If I got your questions right I think the following should do the trick, notice the @dummy field is just to get you one big group for the second GROUPBY
FT.AGGREGATE myIndex "*"
LOAD 1 @timestamp APPLY "@timestamp - (@timestamp % 3600)" AS hour
APPLY 0 as dummy
GROUPBY 2 @hour @dummy
REDUCE COUNT_DISTINCT 1 @user_id AS num_users
GROUPBY 1 @dummy
REDUCE QUANTILE 2 @num_users 0.5 AS median