Few-Shot Prompting: How Many Examples Is Enough?

“How many examples” is two questions wearing one coat. The cost of another example is arithmetic you can do exactly. The benefit depends on what you believe examples are for — and the published evidence on that is more surprising than the advice built on it.

The cost side, exactly

Examples are input tokens on every single call, forever. Take a classification prompt whose examples run about 180 tokens each (input plus label plus the delimiters around them):

shots   prompt tokens   $/call @ $3/M in   $/month @ 200k calls
  0            420            0.00126              $252
  1            600            0.00180              $360
  3            960            0.00288              $576
  5           1320            0.00396              $792
 10           2220            0.00666             $1332

Substitute your own token counts and your provider’s own input price — the numbers above use $3 per million input tokens purely as an illustration, and prices move. The structure is what matters: the marginal cost of a shot is linear in its token length and in your call volume, and it recurs. Ten shots is not “a bigger prompt”, it is roughly five times the input bill of one shot, every month.

Two things change that calculus. Prompt caching, if your examples are a fixed prefix, collapses the recurring cost to the cache-read rate — often around a tenth, which turns the $1,332 row into something near $133 plus write costs. And prefill latency: 2,220 tokens of prompt is measurably slower to first token than 420, which matters if a human is watching.

What examples actually teach

The intuition is that demonstrations teach the mapping — show the model five inputs with correct labels and it learns the rule. Min et al. (2022), Rethinking the Role of Demonstrations, tested that directly by replacing the gold labels in the demonstrations with random ones from the same label set. Performance barely moved. What did matter was the label space, the distribution of the input text, and the overall format of the demonstrations.

Read that result as a design instruction rather than a curiosity. Your examples are doing four jobs, and only the last one is what people think they are doing:

Fixing the output shape — the exact field names, the casing, whether there is prose around the JSON.
Fixing the label set — the model learns the permitted answers are exactly these four strings.
Fixing the granularity — one sentence or three paragraphs, and how much hedging is normal.
Teaching the boundary between confusable cases — the only job that needs the labels to be correct, and the only one that justifies more than one or two shots.

This is also why Reynolds and McDonell (2021) could show carefully framed zero-shot prompts matching few-shot ones: if the demonstrations are mostly locating a task the model already knows, a good description can locate it too.

The shape of the curve, from the literature

Nobody here has run a 0/1/3/5/10 sweep on current models, and a page that claimed to would be lying. What the literature reports is consistent enough to plan around. Brown et al. (2020) — the GPT-3 paper, which is where the term few-shot prompting comes from — published accuracy against shot count from zero up to the tens across many tasks. The characteristic shape is a large jump from zero to one, a smaller one from one to a handful, and a long flat stretch after that. The same paper is where the observation that in-context learning improves sharply with model scale comes from, which is why old shot-count advice reads as too high today.

So the honest planning assumption is: most of the available benefit arrives by about three examples, the curve is flat well before ten, and beyond that you are buying variance, not accuracy. Verify it on your own task — a sweep over your own eval set is a two-hour job and settles the question for your data in a way no published curve can.

A decision procedure

Zero when the task is fully describable in a sentence and the model accepts a response schema. The schema does the shape job the examples were doing.
One when there is an output shape you cannot enforce mechanically. One correct exemplar is worth more than a paragraph of description.
Three to five when there is a label set with genuinely confusable boundaries. Budget roughly one example per confusable pair, and use the examples on the pairs you see failing.
More than ten only when the examples are selected per request by similarity search. At that point it is retrieval, it has a retrieval’s failure modes, and the shot count is set by your context budget rather than by pedagogy.

Choosing and ordering the examples

Two published effects should change how you assemble the block, and both are easy to trip over.

Order is not neutral. Lu et al. (2022), Fantastically Ordered Prompts and Where to Find Them, showed that permuting the same set of demonstrations can move accuracy across an enormous range on the same task and model — from near state-of-the-art to near chance in their experiments. If your prompt is assembled by iterating a dictionary, your accuracy has an implementation detail in it.

Label balance is not neutral either. Zhao et al. (2021), Calibrate Before Use, identified three biases in few-shot prompting: majority-label bias, recency bias (the last example counts for more), and common-token bias. Five examples that all end in approve shift the model toward approve regardless of the input.

Practically:

Balance the label distribution, and do not end on the same label twice.
Pin the order in code and treat it as part of the prompt version. A reordered example block is a prompt change and deserves an eval run.
Use edge cases, not typical cases. The typical case is what the model already gets right.
Review every example for correctness once a quarter. A wrong label in an exemplar is a bug that reproduces on every request and never throws.

The whole cost table collapses if the example block is genuinely being cached, and that is checkable rather than assumable: an OpenAI-compatible response reports usage.prompt_tokens_details.cached_tokens, which the API documentation describes alongside the rest of the usage object. If that number is zero on every call, something above your examples is varying.