🚀 Meet RyanCodrai/turbovec: a gem from today's GitHub trending list.

🔗 https://github.com/RyanCodrai/turbovec
📝 A vector index built on TurboQuant, written in Rust with Python bindings
──────────────────────────────

What is turbovec?
is a fast, memory‑efficient vector search library written in Rust with Python bindings. It implements Google Research’s TurboQuant algorithm – a data‑oblivious quantizer that needs no separate training phase and delivers near‑optimal distortion.

Why you’ll care
- A 10 M‑document float‑32 corpus (~31 GB) fits in ~4 GB of RAM.
- Search is consistently faster than FAISS IndexPQFastScan (≈3.4× speed‑up at 4‑bit, ≈20‑30 % at 2‑bit).
- No “train‑then‑load” step – you can add vectors on the fly.
- Incremental, crash‑safe persistence (`sync`) writes only what changed.
- Built‑in filtering lets you restrict searches to an allow‑list without extra post‑processing.
- Pure‑local deployment – perfect for privacy‑sensitive or latency‑critical RAG pipelines.

Key features at a glance
- Online ingest: `add()` vectors anytime; no rebuilding.
- SIMD‑optimized search: hand‑written kernels (NEON SDOT/SMMLA, AVX‑512 VNNI, AVX2, scalar fallback).
- Incremental saves: `sync(path)` persists deltas with a single fsync; full snapshots still available via `write`/`load`.
- Filter‑aware search: pass an id allowlist or slot bitmask; the kernel skips irrelevant blocks.
- Stable external IDs: `IdMapIndex` keeps your own uint64 identifiers and supports O(1) deletes.
- Framework adapters: drop‑in replacements for LangChain, LlamaIndex, Haystack, Agno.

Getting started – Python

pip install turbovec

from turbovec import TurboQuantIndex

# create a 1536‑dim index, 4‑bit quantization
index = TurboQuantIndex(dim=1536, bit_width=4)

# add vectors (numpy float32, shape (n, dim))
index.add(vectors)
index.add(more_vectors)

# search
scores, ids = index.search(query, k=10)

# persistence
index.write("my_index.tv")          # full snapshot
index.sync("my_index.tv")           # incremental, crash‑safe
loaded = TurboQuantIndex.load("my_index.tv")

Stable IDs example

from turbovec import IdMapIndex
import numpy as np

idx = IdMapIndex(dim=1536, bit_width=4)
idx.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64))

scores, external_ids = idx.search(query, k=10)
idx.remove(1002)                     # O(1) delete by id
idx.sync("my_index.tvim")

Hybrid (filtered) search – combine a coarse external retriever with dense reranking:

allowed = np.array(db.execute(
    "SELECT id FROM docs WHERE tenant=?", (t,)
).fetchall(), dtype=np.uint64)

scores, ids = idx.search(query, k=10, allowlist=allowed)

The filter is evaluated inside the SIMD kernel, so only the allowed blocks incur any computation.

Getting started – Rust

cargo add turbovec

use turbovec::TurboQuantIndex;

let mut index = TurboQuantIndex::new(1536, 4).unwrap();
index.add(&vectors);
let (scores, ids) = index.search(&queries, 10);
index.write("index.tv").unwrap();
let loaded = TurboQuantIndex::load("index.tv").unwrap();

(1/2)

Technical highlights
- TurboQuant provides data‑oblivious quantization with near‑optimal distortion and no training overhead.
- SIMD kernels operate on a vector‑major layout, allowing direct dot‑product computation without costly transposes.
- On ARM, kernels use NEON SDOT/SMMLA; on x86 they leverage AVX‑512 VNNI and `vpermb`.
- Benchmarks (100 K vectors, 1 K queries, k = 64) show median single‑thread speeds 3.4× faster than FAISS at 4‑bit and 20‑30 % faster at 2‑bit across both architectures.
- Insertion latency per vector is 6‑20 µs (≈8‑14× faster than FAISS), and deletions are O(1) at sub‑microsecond cost.
- Compression plots demonstrate up to 8× reduction in RAM vs raw float32.

Who should use turbovec?
- Engineers building Retrieval‑Augmented Generation (RAG) systems where memory, latency, or data‑privacy are critical.
- Teams that need a drop‑in FAISS alternative but want better speed and smaller footprints.
- Rust or Python developers who prefer a single‑library solution with native SIMD performance.
- Anyone integrating vector stores into LangChain, LlamaIndex, Haystack, or custom pipelines.

One‑liner takeaway
lets you store massive embedding collections in a few gigabytes and search them faster than FAISS – all while staying completely local.

──────────────────────────────
🧠 Channel: https://t.me/GithubRe
(2/2)