Recern Vector 0.4.0 · Measured 2026-11-02

Recall, latency and storage.

We measured the released Python package on full SIFT1M and GloVe-100: embedded f32 and int8 collections against faiss HNSW. Each search setting has three timed repetitions over the same 1,000 test queries. These results measure one query at a time, not an HTTP server or a hosted service.

Apple M3 Max · 36 GiB RAM · macOS 27.0.1 · Python 3.13.15
Recern Vector 0.4.0 · faiss-cpu 1.15.1 · NumPy 2.5.3 · h5py 3.16.0

7,712/s

queries/s on SIFT1M at recall ≥ 0.95

618/s

queries/s on GloVe-100 at recall ≥ 0.95

102.7s

to build the SIFT1M f32 index · 4 threads

Same recall target

Throughput at recall@10 ≥ 0.95

Each cell selects that engine’s fastest tested setting reaching the target. A missing result means the target was not reached within the measured ef sweep.

Engine	SIFT1M · recall ≥ 0.95	GloVe-100 · recall ≥ 0.95
Recern Vector 0.4.0 · f32HNSW · 0.4.0	7,712queries/s · ef=80	618queries/s · ef=1280
Recern Vector 0.4.0 · int8HNSW · 0.4.0	3,565queries/s · ef=80	246queries/s · ef=1280
faiss HNSW · f32HNSW · 1.15.1	6,605queries/s · ef=80	356queries/s · ef=1280

The complete trade-off

Recall and single-query throughput

Both f32 and int8 are evaluated against the original dataset’s published neighbours. The int8 results include quantization loss. Query throughput is the median of three repeated loops, not a claim of concurrent serving capacity.

Recern f32 HNSW m=16
Recern int8 HNSW m=16
faiss f32 HNSW m=16

Recall@10 on the first 1,000 test queries; throughput of one query at a time from Python, on a log scale. The recall axis is stretched toward 1.0 so the high-recall region is readable; exact search sits at the right edge. Settings below recall 0.5 are not shown. The same numbers are in the tables below.

Three recall levels

Choose a target before comparing speed

QPS and p95 belong to the same selected setting. A higher ef can improve recall while reducing throughput; no interpolation or extrapolation is used.

SIFT1M · 1,000,000 × 128 · l2

Engine	Recall ≥ 0.90	Recall ≥ 0.95	Recall ≥ 0.99
Recern Vector 0.4.0 · f32HNSW m=16 · ef construction=200	14,681queries/s · ef=40p95 0.086 ms	7,712queries/s · ef=80p95 0.164 ms	4,083queries/s · ef=160p95 0.31 ms
Recern Vector 0.4.0 · int8HNSW m=16 · ef construction=200	6,126queries/s · ef=40p95 0.2 ms	3,565queries/s · ef=80p95 0.339 ms	not reached
faiss HNSW · f32HNSW m=16 · ef construction=200	12,903queries/s · ef=40p95 0.095 ms	6,605queries/s · ef=80p95 0.192 ms	3,802queries/s · ef=160p95 0.316 ms

GloVe-100 · 1,183,514 × 100 · cosine

Engine	Recall ≥ 0.90	Recall ≥ 0.95	Recall ≥ 0.99
Recern Vector 0.4.0 · f32HNSW m=16 · ef construction=200	2,251queries/s · ef=320p95 0.552 ms	618queries/s · ef=1280p95 2 ms	not reached
Recern Vector 0.4.0 · int8HNSW m=16 · ef construction=200	930queries/s · ef=320p95 1.425 ms	246queries/s · ef=1280p95 5.513 ms	not reached
faiss HNSW · f32HNSW m=16 · ef construction=200	1,623queries/s · ef=320p95 0.782 ms	356queries/s · ef=1280p95 3.317 ms	not reached

Index construction and files

Build time and storage

All three engines use 4 build threads, m=16 and ef construction=200. Build times are one index build per configuration. Recern save and checkpoint are recorded separately; faiss writes an index file and does not provide the same database/WAL durability semantics.

SIFT1M

Engine	Build	Save / write	Checkpoint	Reopen	File size
Recern Vector 0.4.0 · f32	102.69 s	5.46 s	7.69 s	5.53 s	630.8 MiB
Recern Vector 0.4.0 · int8	299.01 s	2.39 s	3.39 s	2.74 s	268.4 MiB
faiss HNSW · f32	80.51 s	0.13 s	—	0.1 s	625.9 MiB

GloVe-100

Engine	Build	Save / write	Checkpoint	Reopen	File size
Recern Vector 0.4.0 · f32	149.44 s	5.19 s	7.42 s	5.63 s	620.3 MiB
Recern Vector 0.4.0 · int8	387.87 s	2.52 s	3.61 s	3.04 s	286.2 MiB
faiss HNSW · f32	118.25 s	0.16 s	—	0.1 s	614.3 MiB

File size is the checkpoint plus any WAL for Recern, and the serialized index for faiss. Formats include different features and overhead. This is not peak RAM usage. ID preparation and faiss cosine normalization are outside the index build timer; Recern normalizes inputs inside its API.

Method and boundaries

What was measured

SIFT1M: 1,000,000 vectors, 128 dimensions, L2. GloVe-100: 1,183,514 vectors, 100 dimensions, cosine.
The first 1,000 test queries, k=10, with the published ANN-Benchmarks ground truth. Recern uses string IDs and faiss uses positional IDs.
ef: 10, 20, 40, 80, 160, 320, 640, 1280. Each repetition warms up on 100 queries, then uses a reproducibly shuffled query order. Search uses one thread and one caller.
Latency includes the Python call through returned IDs; recall calculation is outside the timer. p50 and p95 are medians of per-repetition percentiles.
Graphs are built once, then saved and reopened. Parallel graph construction can vary between runs. The reported throughput ranges below show query-loop variation, not build-to-build confidence intervals.
Measured on a desktop workstation, without process or CPU isolation. Other activity, OS scheduling and thermal conditions can affect the results.
No filters, document embedding model, metadata payload workload, HTTP, Qdrant, multi-client load, crash test or larger-than-memory operation is measured here.

Every measured setting

Inspect the raw measurements

The JSON files retain every repetition, dataset hashes, exact package versions and the installed native-binary hash.

SIFT1M · all ef settings and repeated-loop ranges

Engine	ef	Recall@10	QPS	QPS range	p50 ms	p95 ms
Recern Vector 0.4.0 · f32	10	0.7534	34,975	31,591–36,228	0.027	0.041
Recern Vector 0.4.0 · f32	20	0.8635	23,442	20,359–24,645	0.041	0.059
Recern Vector 0.4.0 · f32	40	0.9432	14,681	12,728–14,908	0.068	0.086
Recern Vector 0.4.0 · f32	80	0.9829	7,712	7,501–8,270	0.13	0.164
Recern Vector 0.4.0 · f32	160	0.9956	4,083	4,073–4,210	0.246	0.31
Recern Vector 0.4.0 · f32	320	0.9986	2,229	2,212–2,384	0.45	0.565
Recern Vector 0.4.0 · f32	640	0.9990	1,321	1,267–1,367	0.764	0.978
Recern Vector 0.4.0 · f32	1280	0.9991	749	743–749	1.374	1.625
Recern Vector 0.4.0 · int8	10	0.7497	14,943	14,492–15,169	0.065	0.094
Recern Vector 0.4.0 · int8	20	0.8585	9,929	9,838–9,972	0.101	0.13
Recern Vector 0.4.0 · int8	40	0.9353	6,126	6,070–6,128	0.168	0.2
Recern Vector 0.4.0 · int8	80	0.9711	3,565	3,527–3,578	0.29	0.339
Recern Vector 0.4.0 · int8	160	0.9846	1,985	1,953–1,990	0.522	0.628
Recern Vector 0.4.0 · int8	320	0.9868	1,110	1,075–1,119	0.93	1.145
Recern Vector 0.4.0 · int8	640	0.9870	632	625–636	1.635	2.041
Recern Vector 0.4.0 · int8	1280	0.9870	359	356–360	2.879	3.644
faiss HNSW · f32	10	0.6987	33,932	33,141–34,131	0.029	0.041
faiss HNSW · f32	20	0.8285	19,515	18,223–22,021	0.05	0.069
faiss HNSW · f32	40	0.9245	12,903	12,881–12,945	0.078	0.095
faiss HNSW · f32	80	0.9756	6,605	6,531–7,117	0.151	0.192
faiss HNSW · f32	160	0.9938	3,802	3,743–3,803	0.271	0.316
faiss HNSW · f32	320	0.9981	1,947	1,942–1,950	0.527	0.61
faiss HNSW · f32	640	0.9990	941	939–942	1.091	1.248
faiss HNSW · f32	1280	0.9991	405	399–407	2.498	2.869

Download SIFT1M JSON →

Dataset SHA-256: dd6f0a6ed6b7ebb8934680f861a33ed01ff33991eaee4fd60914d854a0ca5984

GloVe-100 · all ef settings and repeated-loop ranges

Engine	ef	Recall@10	QPS	QPS range	p50 ms	p95 ms
Recern Vector 0.4.0 · f32	10	0.5220	34,717	33,756–35,509	0.027	0.042
Recern Vector 0.4.0 · f32	20	0.6416	22,104	21,614–22,154	0.043	0.065
Recern Vector 0.4.0 · f32	40	0.7398	13,254	13,098–13,275	0.073	0.105
Recern Vector 0.4.0 · f32	80	0.8132	7,683	7,659–7,700	0.128	0.172
Recern Vector 0.4.0 · f32	160	0.8701	4,218	4,157–4,248	0.238	0.299
Recern Vector 0.4.0 · f32	320	0.9128	2,251	2,235–2,255	0.449	0.552
Recern Vector 0.4.0 · f32	640	0.9455	1,154	1,067–1,193	0.872	1.094
Recern Vector 0.4.0 · f32	1280	0.9699	618	600–620	1.648	2
Recern Vector 0.4.0 · int8	10	0.5197	13,312	11,146–13,599	0.07	0.114
Recern Vector 0.4.0 · int8	20	0.6289	8,603	7,339–8,709	0.11	0.173
Recern Vector 0.4.0 · int8	40	0.7318	5,159	5,079–5,363	0.185	0.291
Recern Vector 0.4.0 · int8	80	0.8093	3,076	3,029–3,144	0.32	0.447
Recern Vector 0.4.0 · int8	160	0.8644	1,717	1,710–1,765	0.586	0.773
Recern Vector 0.4.0 · int8	320	0.9052	930	925–962	1.09	1.425
Recern Vector 0.4.0 · int8	640	0.9357	512	510–516	1.988	2.548
Recern Vector 0.4.0 · int8	1280	0.9603	246	244–246	4.144	5.513
faiss HNSW · f32	10	0.4689	28,233	27,914–28,554	0.033	0.054
faiss HNSW · f32	20	0.6036	17,597	17,182–17,900	0.054	0.085
faiss HNSW · f32	40	0.7071	10,641	10,623–10,690	0.09	0.136
faiss HNSW · f32	80	0.7917	5,637	5,560–6,016	0.174	0.243
faiss HNSW · f32	160	0.8523	3,192	3,188–3,282	0.313	0.399
faiss HNSW · f32	320	0.9003	1,623	1,548–1,693	0.62	0.782
faiss HNSW · f32	640	0.9371	801	794–819	1.261	1.53
faiss HNSW · f32	1280	0.9619	356	336–357	2.837	3.317

Download GloVe-100 JSON →

Dataset SHA-256: 544af1d5e84e112cd4749571dcfd8ca109818a572f850af75a3a09e093a953c4

Reproduce

Run the same harness

The current harness records macOS hardware metadata. Use a fresh work directory for each run. Large datasets and indexes require several gigabytes of free disk space; compare machines and builds explicitly.

Benchmark script · Pinned requirements

# macOS, Python 3.13; run in a fresh directory
curl --fail --location -O https://recern.net/benchmarks/release_bench.py
curl --fail --location -O https://recern.net/benchmarks/requirements-0.4.0.txt
python3.13 -m venv .bench-venv
.bench-venv/bin/python -m pip install -r requirements-0.4.0.txt
mkdir bench-data
curl --fail --location https://ann-benchmarks.com/sift-128-euclidean.hdf5 -o bench-data/sift-128-euclidean.hdf5
curl --fail --location https://ann-benchmarks.com/glove-100-angular.hdf5 -o bench-data/glove-100-angular.hdf5

OPENBLAS_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 OMP_NUM_THREADS=4 \
  .bench-venv/bin/python release_bench.py sift --data-dir bench-data \
  --output results/sift.json --work work/sift --queries 1000 --repeats 3 --build-threads 4
OPENBLAS_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 OMP_NUM_THREADS=4 \
  .bench-venv/bin/python release_bench.py glove --data-dir bench-data \
  --output results/glove.json --work work/glove --queries 1000 --repeats 3 --build-threads 4

Harness SHA-256: 46d61eaffa74ee1ec11bd91b14c84637702c5a40eb6095eb5c0a276d15335aad

Earlier work

Keep historical results in context

The pre-0.2.0 report from 7 October 2026 compared the prototype with faiss, LanceDB and sqlite-vec using a different build protocol. It is preserved separately; it is not mixed into these release measurements.

Read the historical report →