A new way to adapt
Traditionally you adapt a model by fine tuning its weights on a labeled set. Large language models opened another route: in context learning, where you simply place a few worked examples in the prompt and the model infers the pattern, updating no weights at all.
Zero, one, and few shot
The number of examples in the prompt names the setting:
- Zero shot gives only an instruction and no examples
- One shot includes a single demonstration
- Few shot includes several demonstrations of input and desired output
From these examples the model picks up the format, the task, and the style, then applies them to a new query. It is learning during inference, drawing on patterns absorbed in pretraining.
Strengths and limits
In context learning needs no training run and adapts instantly, which is ideal for quick tasks. But it is limited by the context window, sensitive to example order and wording, and usually less reliable than true fine tuning for hard, high volume tasks.
Key idea
Few shot in context learning adapts a model by showing examples in the prompt rather than changing weights, trading instant flexibility for context limits and prompt sensitivity.