LearnViz CompSci
Home / Learn / Math for Computing / Maximum Likelihood Estimation

Advanced Topic 48 of 51 Math for Computing

Maximum Likelihood Estimation

Choosing parameters that make data most probable.

How it works

Maximum likelihood estimation chooses the parameter values that make the observed data most probable under a model. The likelihood is the probability of the data as a function of the parameters, and for independent data it is a product, so the log-likelihood, a sum, is maximized by setting the derivative to zero or using numerical optimization. Estimators are consistent and asymptotically normal under regularity conditions, with the Fisher information giving their variance. Minimizing cross-entropy or squared error in many models is equivalent to maximum likelihood.

Where it is used

Fitting distributions, logistic regression, training neural networks with cross-entropy, estimating parameters in time series and survival models, and hidden Markov models use maximum likelihood.

Common pitfall

With small samples it can be biased or overfit, such as the variance estimate dividing by n, and it may give extreme values like zero probabilities for unseen events, requiring smoothing or priors. Likelihood surfaces can have multiple local maxima. Check model assumptions and use regularization or Bayesian methods with limited data.

Where it fits in the course

Maximum Likelihood Estimation is topic 48 of 51 in Math for Computing, rated Advanced. It follows Stochastic Processes. The next topic is Bayesian Inference.

Practice and explore

How to study this topic

  1. Read the explanation above and restate the definition in your own words.
  2. Open the interactive lesson and answer its mini quiz.
  3. Read the common pitfall, then write a short example that avoids it.
  4. Check your understanding in the Math for Computing quiz, and revisit the topic if you miss a question.

Frequently asked questions

What is Maximum Likelihood Estimation?
Choosing parameters that make data most probable. It belongs to Math for Computing and is an advanced-level topic.
Where do people use Maximum Likelihood Estimation?
Fitting distributions, logistic regression, training neural networks with cross-entropy, estimating parameters in time series and survival models, and hidden Markov models use maximum likelihood.
What is the most common mistake with Maximum Likelihood Estimation?
With small samples it can be biased or overfit, such as the variance estimate dividing by n, and it may give extreme values like zero probabilities for unseen events, requiring smoothing or priors.
What should I learn before this topic?
Start with Stochastic Processes, the topic just before it in Math for Computing.
How can I practice it?
Answer the mini quiz in the interactive lesson, then take the Math for Computing quiz.
Is Maximum Likelihood Estimation hard to learn?
It is an Advanced topic. Expect to read the explanation more than once and to connect it with earlier topics. Working through a small example by hand helps most.
How long does it take to learn Maximum Likelihood Estimation?
Most learners can read the explanation and finish the interactive lesson in under an hour. Fluency comes from applying the idea, so plan a few short practice sessions with the quiz.
Which topics are related to Maximum Likelihood Estimation?
Stochastic Processes, Bayesian Inference, Monte Carlo Methods are closely connected. The full list is in the related topics section of this page.
What should I study after Maximum Likelihood Estimation?
Continue with Bayesian Inference, the next topic in Math for Computing.

Related topics