⚡️ How Redis counts billions of unique values while barely using memory
There's an algorithm called HyperLogLog. It allows you to roughly estimate how many unique elements have passed through the system, using about 12 KB of memory.
The idea is simple: Redis doesn't store the elements themselves.
It does the following:
- Takes an element
- Calculates a hash from it
- Uses part of the hash as a cell number
- Checks the other part to see how many consecutive zeros it contains
- If the new number is larger than the old one, it updates the cell
Why does this work?
Because a long series of zeros in the hash is rare.
For example:
- 1 consecutive zero - quite common
- 5 consecutive zeros - less common
- 10 consecutive zeros - about a 1 in 1024 chance
- 20 consecutive zeros - a very rare event
If Redis sees a very rare pattern, it means that many different elements have likely passed through it.
Redis uses 16,384 small counters. Each stores the maximum "rarity" it has seen for its group of elements.
Then Redis combines these values mathematically to get an estimate of unique elements.
Not an exact number, but a very close approximation.
The main trick of HyperLogLog:
it can handle millions or even billions of values, but memory hardly increases at all.
That's why Redis can count unique users, IPs, requests, or events without huge tables and lists.
#Redis #HyperLogLog #DataScience #Tech #BigData #MemoryEfficiency
✨ Join Best TG Channels https://t.me/addlist/0f6vfFbEMdAwODBk
⭐️ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
🚀 Level up your AI & Data Science skills with HelloEncyclo — a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
✅ 13 courses live + 40+ coming soon
🎯 One access, lifetime updates
🔑 Use code: PRESALE-BOOK-WAVE-2GFG
👉 https://helloencyclo.com/?ref=HUSSEINSHEIKHO
There's an algorithm called HyperLogLog. It allows you to roughly estimate how many unique elements have passed through the system, using about 12 KB of memory.
The idea is simple: Redis doesn't store the elements themselves.
It does the following:
- Takes an element
- Calculates a hash from it
- Uses part of the hash as a cell number
- Checks the other part to see how many consecutive zeros it contains
- If the new number is larger than the old one, it updates the cell
Why does this work?
Because a long series of zeros in the hash is rare.
For example:
- 1 consecutive zero - quite common
- 5 consecutive zeros - less common
- 10 consecutive zeros - about a 1 in 1024 chance
- 20 consecutive zeros - a very rare event
If Redis sees a very rare pattern, it means that many different elements have likely passed through it.
Redis uses 16,384 small counters. Each stores the maximum "rarity" it has seen for its group of elements.
Then Redis combines these values mathematically to get an estimate of unique elements.
Not an exact number, but a very close approximation.
The main trick of HyperLogLog:
it can handle millions or even billions of values, but memory hardly increases at all.
That's why Redis can count unique users, IPs, requests, or events without huge tables and lists.
#Redis #HyperLogLog #DataScience #Tech #BigData #MemoryEfficiency
✨ Join Best TG Channels https://t.me/addlist/0f6vfFbEMdAwODBk
⭐️ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
🚀 Level up your AI & Data Science skills with HelloEncyclo — a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
✅ 13 courses live + 40+ coming soon
🎯 One access, lifetime updates
🔑 Use code: PRESALE-BOOK-WAVE-2GFG
👉 https://helloencyclo.com/?ref=HUSSEINSHEIKHO
❤1
Forwarded from Machine Learning with Python
🚨 ONLY THE FIRST 5 GET THIS.
I'm sharing this link with my network once — and only the first 5 people who enroll through it lock in a deal that has never been offered before.
👑 Lifetime access to HelloEncyclo — every AI, ML & Data Science course ever built — for ~$41. Once. Forever.
This isn't a drill. This isn't a rerun.
This is the founding-member price — and it disappears the moment the first 250 seats globally are gone.
✅ 13 courses live right now
✅ 40+ more in 2–3 weeks
✅ Every future course included automatically
✅ 15-day money-back — full refund, no questions
Code: PRESALE-BOOK-WAVE-2GFG
(Log in with Gmail · valid once · applies at checkout)
👇 First 5. That's it.
https://helloencyclo.com/?ref=HUSSEINSHEIKHO
⏳ Once those 5 seats go through this link —
I'm not sharing it again. 🔥
I'm sharing this link with my network once — and only the first 5 people who enroll through it lock in a deal that has never been offered before.
👑 Lifetime access to HelloEncyclo — every AI, ML & Data Science course ever built — for ~$41. Once. Forever.
This isn't a drill. This isn't a rerun.
This is the founding-member price — and it disappears the moment the first 250 seats globally are gone.
✅ 13 courses live right now
✅ 40+ more in 2–3 weeks
✅ Every future course included automatically
✅ 15-day money-back — full refund, no questions
Code: PRESALE-BOOK-WAVE-2GFG
(Log in with Gmail · valid once · applies at checkout)
👇 First 5. That's it.
https://helloencyclo.com/?ref=HUSSEINSHEIKHO
⏳ Once those 5 seats go through this link —
I'm not sharing it again. 🔥
This media is not supported in your browser
VIEW IN TELEGRAM
🎰 Welcome Bonus 1200% — Maczo Crypto Casino
🎮 Crypto exchange · Sports · Live casino — all in one place
💳 USDT instant deposit & withdrawal
→https://tglink.io/10ac0a48667b40
→ Affiliate 60%
🎮 Crypto exchange · Sports · Live casino — all in one place
💳 USDT instant deposit & withdrawal
→https://tglink.io/10ac0a48667b40
→ Affiliate 60%