Validation data
Data used to assess and select model variants during development.
What is validation data for?
Validation data helps assess and select model variants or settings during development. It has a different role from the examples through which a model learns its parameters. It may be used as a separate held-out dataset or through changing partitions. Suitable splitting also depends on temporal or other dependencies between examples. Model selection must be accounted for when arranging a final independent assessment.
In a fictional spare-part image-classification project, the team plans to compare several variants using validation images. It examines errors by item class and capture conditions instead of selecting by an overall score alone. In this project, the images reserved for final testing remain outside that selection. Repeatedly tailoring variants to the same validation cases can produce an overly favourable picture there too. The team therefore documents the roles of the images and the selection decisions. A good validation score alone does not establish equally good performance on later product photos.
Sources
Related terms
- Machine learningModels learn patterns from data and apply them to new inputs.
- LLM evaluationLLM evaluation tests model outputs against defined tasks and criteria. One good example does not establish overall quality.
- Training dataThe examples from which a model learns relationships or parameters during training.
- Test data in machine learningData reserved for final assessment whose results were not used to select the model variant being assessed.
- OverfittingExcessive adaptation to details of training data that reduces a model’s ability to generalise to new examples.
- UnderfittingA model fails to capture the relationships needed for its task adequately and therefore produces weak predictions.
Your project with ASES
Describe your project to us. We will get back to you with an initial assessment.