LearnViz CompSci
Home / Learn / AI & Machine Learning / Training and Test Sets

Beginner Topic 6 of 51 AI & Machine Learning

Training and Test Sets

Separating data used to learn from data used to evaluate.

How it works

To estimate how a model will perform on new data, the dataset is split into a training set used to fit parameters, often a validation set used to tune hyperparameters and choose models and a test set held out until the end for an unbiased final estimate. Typical splits are 70/15/15 or 80/10/10, with stratification to keep class proportions and time-based splits for temporal data. The test set must never influence training or tuning decisions.

Step-by-step walkthrough

  1. Split the data before training.
  2. Fit the model on the training set.
  3. Tune settings on the validation set.
  4. Evaluate once on the untouched test set.
  5. This estimates how it will perform on new data, and test leakage gives falsely good scores.

Where it is used

Every machine-learning project uses splits to detect overfitting and to compare models honestly. Competitions use hidden test sets to maintain integrity.

Common pitfall

Data leakage, where information from the test set sneaks into training through preprocessing fitted on all data, duplicates across splits or future information, gives overly optimistic results. Repeatedly tuning on the test set turns it into a validation set. Random splits fail for grouped or time-series data. Fit scalers and encoders on training data only and keep the test set untouched.

Where it fits in the course

Training and Test Sets is topic 6 of 51 in AI & Machine Learning, rated Beginner. It follows Reinforcement Learning Basics. The next topic is Features and Labels.

Practice and explore

How to study this topic

  1. Read the explanation above and restate the definition in your own words.
  2. Open the interactive lesson and answer its mini quiz.
  3. Read the common pitfall, then write a short example that avoids it.
  4. Check your understanding in the AI & Machine Learning quiz, and revisit the topic if you miss a question.

Frequently asked questions

What is Training and Test Sets?
Separating data used to learn from data used to evaluate. It belongs to AI & Machine Learning and is a beginner-level topic.
Where do people use Training and Test Sets?
Every machine-learning project uses splits to detect overfitting and to compare models honestly.
What is the most common mistake with Training and Test Sets?
Data leakage, where information from the test set sneaks into training through preprocessing fitted on all data, duplicates across splits or future information, gives overly optimistic results.
What should I learn before this topic?
Start with Reinforcement Learning Basics, the topic just before it in AI & Machine Learning.
How can I practice it?
Answer the mini quiz in the interactive lesson, then take the AI & Machine Learning quiz.
Is Training and Test Sets hard to learn?
It is a Beginner topic, so it works as an entry point. If the terms are new, read the explanation twice and try the visualizer before the quiz.
How long does it take to learn Training and Test Sets?
Most learners can read the explanation and finish the interactive lesson in under an hour. Fluency comes from applying the idea, so plan a few short practice sessions with the quiz.
Which topics are related to Training and Test Sets?
Reinforcement Learning Basics, Features and Labels, Unsupervised Learning are closely connected. The full list is in the related topics section of this page.
What should I study after Training and Test Sets?
Continue with Features and Labels, the next topic in AI & Machine Learning.

Related topics