Temperature in language models
A setting that influences the distribution of possible next tokens during sampling and therefore their selection.
What does temperature change in generated text?
During language-model sampling, temperature influences how strongly selection concentrates on more likely next tokens. Lower values typically produce a more concentrated distribution, while higher values flatten it and allow more variety in selection. This affects output generation, rather than retraining the model. Supported values and interactions with other selection rules depend on the model. A low temperature establishes neither business correctness nor reliably correct customer matching.
In a fictional project, a model is to suggest a short internal subject line from a service description. For a planned comparison, the model version, instruction and input stay the same; initially, only temperature changes. Reviewers assess clarity, relevance to the request and unsupported additions. More varied wording may help; an invented fault cause would not. Even at very low temperature, outputs can differ across repeated generations. The chosen setting is therefore documented alongside the other conditions and review results.
Sources
Related terms
- Prompt engineeringDeliberately designing and checking tasks, context and output rules for a generative model.
- LLM evaluationLLM evaluation tests model outputs against defined tasks and criteria. One good example does not establish overall quality.
- Confidence scoreA confidence score assesses a model prediction according to a particular method. It does not grant business approval.
- Top-p samplingSelecting the next token from a small set of likely candidates that together cover a probability threshold.
- Reproducibility of AI outputsBeing able to repeat AI generation under documented conditions and compare results against defined criteria.
- Re-rankingRe-ranking reassesses search candidates already retrieved. Relevance to the query and the limits of the initial selection both matter.
Your project with ASES
Describe your project to us. We will get back to you with an initial assessment.