Test data in machine learning
Data reserved for final assessment whose results were not used to select the model variant being assessed.
What should test data show?
Test data is used to assess a selected model with examples that did not guide its development or selection. It should fit the intended use and have an appropriate separation from development data. How to split it also depends on groups and temporal relationships. The name test dataset alone establishes neither independence nor sufficient coverage. The result applies to the data and conditions actually assessed.
In a fictional spare-part project, photos of other physical specimens from the same item classes are reserved for final testing. Closely similar images of the same specimen should not unknowingly be spread across training and testing. Planned lighting and capture conditions are considered as well. The team records which variant and data are assessed. If it then uses the test results to select the next variant, that test has influenced development. A new independent final assessment needs a suitable evaluation outside that selection again. A favourable result with the previous images does not guarantee equally good performance under other conditions.
Sources
Related terms
- Machine learningModels learn patterns from data and apply them to new inputs.
- LLM evaluationLLM evaluation tests model outputs against defined tasks and criteria. One good example does not establish overall quality.
- Computer visionMethods for extracting information from images and video, such as classifying objects, locating them or distinguishing image regions.
- Training dataThe examples from which a model learns relationships or parameters during training.
- Validation dataData used to assess and select model variants during development.
- OverfittingExcessive adaptation to details of training data that reduces a model’s ability to generalise to new examples.
Your project with ASES
Describe your project to us. We will get back to you with an initial assessment.