Learning without weight updates
Large language models can adapt to a task at inference time by reading examples placed in the prompt. This is in context learning: you show a few input and output pairs, then a new input, and the model continues the pattern. No weights change.
Zero, one, and few shot
- Zero shot gives only an instruction and the new input.
- One shot adds a single worked example.
- Few shot provides several examples to make the pattern clearer.
More and clearer examples usually improve results until the context fills up.
How the prompt is built
Strengths and limits
In context learning is fast and flexible since it needs no training run, and it is ideal for quick prototyping. But it consumes context length, can be sensitive to example order and wording, and does not permanently teach the model. For lasting or large scale adaptation, fine tuning is preferred.
Key idea
Few shot in context learning adapts a model by placing examples in the prompt at inference time, trading permanence and context budget for speed and flexibility with no weight updates.