Serving millions of records in under a minute

Some services have one job and have to do it fast. This one had two jobs with very different shapes: answer individual lookups in single-digit milliseconds, and materialize an entire multi-million-row, multi-gigabyte dataset in under a minute when a full pull was requested.

The constraint

Optimizing for one pattern usually hurts the other. Tight per-record latency wants small, cached, index-friendly reads. A full pull wants large, sequential, batched I/O. Treating them as the same code path is how you end up slow at both.

Sub-10ms hot paths

For individual lookups the wins were unglamorous and effective: shape the data for the read, keep the hot set in memory, and avoid chatty round-trips. Once the query plan was stable and the working set fit, average response settled into the sub-10ms range and stayed there under load.

The bulk pull

The full pull was a different problem. Instead of millions of small reads, it became a streamed, batched export — read in large chunks, keep memory bounded, and never hold the whole result set at once. The target wasn't lowest latency per record, it was bounded, predictable total time — and it landed under a minute.

What mattered

• Measure first. The bottleneck was never where intuition said it was.
• Separate the access patterns. Two paths, each optimized for its own shape, beat one clever path trying to serve both.
• Bound your memory. Streaming beats buffering the moment the data outgrows a comfortable slice of RAM.

None of this was exotic. It was picking the right shape for each job and refusing to let the two contaminate each other.