Smaller, Cheaper, Better: The Economics of Accuracy

The industry conflates "bigger" with "better." This ignores basic math. Closing the last few percentage points of accuracy through model size costs exponentially more. Meanwhile, running a cheaper model multiple times and voting gets you higher accuracy at a fraction of the cost.

The Condorcet Math

If a model has 92% accuracy and you run it 5 times with majority voting, the probability of a correct majority isn't 92%; it's 99.5%. This is the Condorcet Jury Theorem applied to inference. As long as individual accuracy exceeds 50% and errors are somewhat independent, aggregating predictions crushes variance.

Model Comparison: Single Pass vs. Majority Vote

The smaller model wins on both accuracy and cost. This isn't theoretical. It's mathematical certainty rooted in the Condorcet Jury Theorem.

Cascades and FrugalGPT

Chen, Zaharia & Zou's FrugalGPT takes this further with cascade architectures. Every query hits the cheapest model first. A confidence check evaluates the response. Only low-confidence outputs escalate to expensive models. Research shows this cuts inference costs by up to 98% while matching or exceeding single-model accuracy.

Recent benchmarks on compute-optimal test-time scaling by Ye et al. found that a 7B parameter model using Best-of-N sampling outperformed GPT-4o and o1-preview on MATH-500. The "bigger is better" narrative isn't just expensive; it's wrong.

The superior system isn't a monolithic giant. It's a compound system orchestrating a fleet of models to optimize the cost-accuracy frontier.

Sources