HyperLogLog: Use case discussion

Redis use cases
1

Hi Folks,
Please feel free to share use cases of HyperLogLog in below context.

The HyperLogLog has three main operations:

  1. Add: To add a new element to the set.
  2. Count: To obtain the cardinality of the set.
  3. Merge: To obtain the union of two sets.
1 Like
2

Here’s a use Case: “Track unique search terms on a high-trafficked website”. The HyperLogLog data structure will track the # of unique values added to a (large) dataset without taking up much memory, but at the cost of some accuracy. In many use cases, you might want a ‘reasonable’ estimate of how many unique words are in a large dataset without having to run a crazy long & cpu-intensive query to find out. HyperLogLog would be perfect for that.

3

@dave - Thanks for pointing out. Yes, we can use it to find distinct record in large datasets.

4

Here’s an excellent HyperLogLog talk at RedisConf20 called “Joinability Analysis at Scale with HyperLogLog” by John Myers of Gretel.ai

1 Like
5

@dave - It was awesome presentation on HyperLogLog

1 Like
6

@manjeetkumar53 I use Hyperloglog, Usually counting unique things, for example the number of unique IPs that connected today to your web site, or the number of unique searches that your users performed, requires to remember all the unique elements encountered so far, in order to match the next element with the set of already seen elements, and increment a counter only if the new element was never seen before

1 Like