Eight calls of one answer cost 4.64 times the electricity of one call of eight
There is a standard way to make a small model more accurate. Ask it the same question eight times, then keep the answer that came up most. That is self-consistency. Those eight answers can arrive in one call that makes all eight side by side, or in eight calls that make one each. On A100s, the datacentre graphics chips that run these models, the eight-call version burned four point six four times the electricity. And the eight-call version drew less power while doing it.
Eight answers lift Phi-3-mini from 81.4% to 89.8% on grade-school maths
Asking more than once buys real accuracy. Over five hundred grade-school maths word problems from GSM8K, going from one answer to eight lifted Phi-3-mini from eighty-one point four percent correct to eighty-nine point eight. Qwen two point five went from fifty-one point four percent to sixty-nine point eight. Those are the paper's two models, three point eight billion parameters and one point five billion. Nothing larger was tested. So eight answers earn their place. What changes the bill is how they arrive.
Four ways of getting eight answers generate the same amount of text, within 1%
Eight answers, four ways to get them. One call that makes all eight side by side. Two calls of four. Four calls of two. Eight calls of one, each waiting for the one before it to finish. The paper calls these schedules. The number everybody publishes, eight, is identical across all four. So is the work. The total amount of text generated varies by zero point eight percent across Phi-3-mini's four schedules, and one percent across Qwen two point five's. The only thing that changed is how many times you asked.
The serial schedule draws less power and still costs 4.64 times as much
Watts is how hard the chip is working right now. Joules is watts multiplied by how long you worked it, and the electricity bill is joules. Put a meter on both runs and the serial one looks gentler. Mean power on the A100 falls from one hundred and seventy-seven point eight watts, when all eight answers are asked for at once, to one hundred and thirty-nine point eight watts when the eight answers are asked one at a time. Drawing less power did not help. Eight calls hold the chip five point nine seven times as long as one call does, so the joules pile up four point six four times over. In chip time, a thousand questions go from two point zero nine GPU-hours to twelve point four nine.
A bigger batch costs less for each piece of text and more for each query
A bigger batch puts more candidates through the chip at once. Split that same work across more calls, and the paper names four costs. Each call re-reads the question from the start. Each call pays the software's own setup cost. Each call waits for the chip to finish and report back. And each call hands the chip less work to do at once. Which of those four costs most, the paper does not say, and it says so. A token is the chunk of text a model writes at a time. On Qwen two point five, on the same A100, as the batch grows from one to eight, the energy spent on each question rises. The energy spent on each token falls.
Energy climbs at every split, 1.00 to 1.63 to 2.71 to 4.64
The penalty is gradual. Relative energy runs one, one point six three, two point seven one, and four point six four across the four schedules for Phi-3-mini. Qwen two point five tracks that same curve and ends at four point eight six. Every split you add costs. The wait gets worse faster than the electricity does. At eight calls of one, the slowest answer in twenty takes five point seven seven times as long as the batched one for Phi-3-mini, and six point one two times for Qwen two point five.
All four answer-length groups pay between four and five times the energy
Three separately scheduled jobs per model, on different A100 nodes, reproduce the gap. The Phi-3-mini energy ratio lands between four point four three and four point six four. The Qwen two point five ratio lands between four point eight five and four point eight eight. Sort the same questions into four groups by how long their answers ran, twenty-five questions a group. Every group lands between four and five times. The ratio does not follow length. On SciQ, school science questions answered in a handful of words, the penalty holds at two point five seven times and three point three four times. The chips changed there too, so the paper will not call that a length effect.
A live serving stack already batches for you, so this bill lands on offline jobs
Every number here was measured on one inference stack, Hugging Face Transformers batch generation on a single GPU. That stack has no continuous batching, a serving system that mixes many different people's requests into shared batches. The people holding this bill run big offline scoring jobs, supercomputer batch jobs, and research loops that ask for one answer at a time. Nothing above three point eight billion parameters was tested. And the energy is what the chip itself reports, with idle power left in and the processor, memory and cooling around it left out. The boundary is the graphics chip. The authors say plainly that these ratios should not be assumed to hold on a continuous-batching server or on larger models. That sweep is the one somebody can run next.
The rule this paper hands you is one line and it costs nothing to apply
Skim it. The finding is one table, and the rule is one line. Use the biggest batch that fits in memory, and as few calls as you can. Self-consistency is exactly the case where answers do not depend on each other. A paper that tells you only how many answers it sampled has not told you what they cost. That reporting gap is what outlives these numbers. The paper's own first table names six earlier studies, including Self-Consistency and Large Language Monkeys. All six leave the call count and the energy blank. Sample Count Is Not Enough, by Mobina Kashaniyan and Ali Jannesari at Iowa State University. An arXiv preprint, twenty six zero nine point one nine four nine nine, version two. The paper ships no code.
