Adaptive Index · Query and maintenance
Recall-targeted search and measured partition maintenance in Quiver.
What it does
Quiver is a dynamic approximate nearest-neighbor index with a C++ core and a PyTorch Tensor Python interface. It supports search, insertion, removal, maintenance, and save/load. The research question is how search effort and partition layout can adapt to queries and changing data.
The work builds on Jason Mohoney / Marius's upstream Quiver system. This portfolio's research and implementation should not be interpreted as sole authorship of every underlying indexing idea.
Section sources
README.md
Adaptive search
A fixed nprobe assigns every query a preset partition budget. Adaptive search instead compares an online estimate with a requested recall target and adjusts how much it scans. Easier and harder queries can therefore receive different amounts of work.
The recall target is an estimated or average target, not a guarantee for each query. Parameter validation and path compatibility matter: an optimization available on one search path is not automatically active in serial, worker, and batched modes alike.
Section sources
src/cpp/src/query_coordinator.cpp
Online maintenance
Maintenance waits for a complete observation window and combines partition size, observed hit rates, and measured scan costs. It can split, delete, reassign, and refine partitions according to that cost model rather than using size alone.
The query loop controls work for the current request; maintenance changes the layout for later requests. These are connected but distinct feedback loops, so an improvement in one must be evaluated together with update and maintenance costs.
Section sources
src/cpp/src/maintenance_policies.cppsrc/cpp/include/maintenance_policies.h
Implementation choices
The current build uses C++20, Python, PyTorch, Faiss, and pybind11. CPU float32 vectors and unique nonnegative int64 IDs are the core interface; L2 and inner-product search are supported, and -1 is reserved for padding.
RaBitQ, scan filters, offline budget models, and centroid graphs are explicit options with path constraints. The encoded RaBitQ path is restricted to compatible serial L2 search. Centroid-graph readiness is bound to the source mutation revision; a stale graph needs rebuilding.
Section sources
CMakeLists.txtsrc/cpp/src/query_coordinator.cppsrc/cpp/src/quiver_index.cpp
Attribution and limits
The current repository is private and the reviewed local revision was not available remotely. Local ignored research artifacts are not public evaluation evidence. This account provides no new speedup or recall result.
The library does not provide vector attribute filtering or a multi-node service. Dynamic updates should not be read as a blanket guarantee for arbitrary concurrent mutation, and average recall targets are not per-query accuracy guarantees.
Section sources
README.mdLICENSE