Quantized and batched serving; cost per 1k requests fell 25% at the same quality bar.
See the full storyProof of work
Search results were mediocre and clickthrough had been flat for a year, with nobody owning the number end to end. I took relevance from the ranking model through the evaluation set to the online test, and made every change defensible against a holdout instead of a hunch.
Quantized and batched serving; cost per 1k requests fell 25% at the same quality bar.
Data was untrustworthy and every dashboard disagreed, so meetings turned into arguments about whose number was right. I owned quality and lineage until the numbers meant something — contracts at the boundaries, tests on the tables that mattered, and alerting when freshness slipped.
Grounded answers in retrieval with citation checks; hallucination rate dropped 70% against the eval set.
Everyone wanted 'AI in the product' and nobody had said what good would look like, so we were one demo away from shipping something confidently wrong. I set the evaluation first — the metric, the holdout, the bar to clear — so we could tell real progress from a nice-looking demo.
Built labeling tools and guidelines; throughput doubled and inter-annotator agreement rose enough to trust the data.
Leave every codebase clearer than I found it.
In public
Practical bits
Unlock with a company account to message directly and reveal:
Jump back to the board with this background already filtered in.
