LearnViz CompSci
Home / Learn / AI & Machine Learning / Loss Functions

Intermediate Topic 19 of 51 AI & Machine Learning

Loss Functions

Quantities measuring how wrong predictions are.

How it works

A loss function measures how far a model's predictions are from the targets, giving a single number the training procedure tries to minimize. Common choices include mean squared error and mean absolute error for regression, Huber loss for robustness, binary and categorical cross-entropy for classification, hinge loss for support vector machines and specialized losses for ranking, segmentation and generative models. The choice shapes what the model learns and how gradients behave.

Step-by-step walkthrough

  1. A loss measures how wrong a prediction is.
  2. Squared error is used for regression.
  3. Cross-entropy is used for classification probabilities.
  4. Hinge loss is used by support vector machines.
  5. Training minimizes the average loss.

Where it is used

Every trained model has a loss: cross-entropy for classifiers, MSE for forecasts, contrastive losses for embeddings and adversarial losses for GANs.

Common pitfall

The loss that is easy to optimize may differ from the metric you care about, such as accuracy, which is not differentiable. Squared error is sensitive to outliers. Class imbalance may need weighted losses. Numerical issues such as log of zero need stable implementations like logits with built-in softmax. Choose a loss that matches the task and monitor both loss and metrics.

Where it fits in the course

Loss Functions is topic 19 of 51 in AI & Machine Learning, rated Intermediate. It follows Gradient Descent. The next topic is Regularization.

Practice and explore

How to study this topic

  1. Read the explanation above and restate the definition in your own words.
  2. Open the interactive lesson and answer its mini quiz.
  3. Read the common pitfall, then write a short example that avoids it.
  4. Check your understanding in the AI & Machine Learning quiz, and revisit the topic if you miss a question.

Frequently asked questions

What is Loss Functions?
Quantities measuring how wrong predictions are. It belongs to AI & Machine Learning and is an intermediate-level topic.
Where do people use Loss Functions?
Every trained model has a loss: cross-entropy for classifiers, MSE for forecasts, contrastive losses for embeddings and adversarial losses for GANs.
What is the most common mistake with Loss Functions?
The loss that is easy to optimize may differ from the metric you care about, such as accuracy, which is not differentiable.
What should I learn before this topic?
Start with Gradient Descent, the topic just before it in AI & Machine Learning.
How can I practice it?
Answer the mini quiz in the interactive lesson, then take the AI & Machine Learning quiz.
Is Loss Functions hard to learn?
It is an Intermediate topic. Be comfortable with the Beginner topics in this area first, then read the common pitfall carefully, since it is where most mistakes happen.
How long does it take to learn Loss Functions?
Most learners can read the explanation and finish the interactive lesson in under an hour. Fluency comes from applying the idea, so plan a few short practice sessions with the quiz.
Which topics are related to Loss Functions?
Gradient Descent, Regularization, Cross-Validation are closely connected. The full list is in the related topics section of this page.
What should I study after Loss Functions?
Continue with Regularization, the next topic in AI & Machine Learning.

Related topics