LearnViz CompSci
Home / Learn / AI & Machine Learning / PCA

Intermediate Topic 27 of 51 AI & Machine Learning

PCA

Projecting data onto directions of maximal variance.

How it works

Principal component analysis finds orthogonal directions, the principal components, along which data varies the most, by computing the eigenvectors of the covariance matrix or the singular value decomposition of the centered data. The first component captures the greatest variance, the second the most remaining variance orthogonal to the first, and so on. Projecting onto the top k components reduces dimensionality, and the explained-variance ratio shows how much information is kept.

Step-by-step walkthrough

  1. Centre the data.
  2. Find the direction with the most variance.
  3. The next direction is perpendicular to it.
  4. Project the data onto the top few axes.
  5. Most information is kept in fewer dimensions.

Where it is used

Visualization, noise reduction, decorrelating features, compressing images, preprocessing for regression or clustering and exploring genomics or finance data use PCA.

Common pitfall

Features must be centered and usually scaled, otherwise large-valued features dominate. PCA is linear, so it misses nonlinear structure, and high-variance directions are not necessarily the most predictive. Components are hard to interpret. Outliers can distort directions. Choose the number of components by explained variance or a scree plot, and fit on training data only.

Where it fits in the course

PCA is topic 27 of 51 in AI & Machine Learning, rated Intermediate. It follows Dimensionality Reduction. The next topic is Hyperparameter Tuning.

Practice and explore

How to study this topic

  1. Read the explanation above and restate the definition in your own words.
  2. Open the interactive lesson and answer its mini quiz.
  3. Read the common pitfall, then write a short example that avoids it.
  4. Check your understanding in the AI & Machine Learning quiz, and revisit the topic if you miss a question.

Frequently asked questions

What is PCA?
Projecting data onto directions of maximal variance. It belongs to AI & Machine Learning and is an intermediate-level topic.
Where do people use PCA?
Visualization, noise reduction, decorrelating features, compressing images, preprocessing for regression or clustering and exploring genomics or finance data use PCA.
What is the most common mistake with PCA?
Features must be centered and usually scaled, otherwise large-valued features dominate.
What should I learn before this topic?
Start with Dimensionality Reduction, the topic just before it in AI & Machine Learning.
How can I practice it?
Answer the mini quiz in the interactive lesson, then take the AI & Machine Learning quiz.
Is PCA hard to learn?
It is an Intermediate topic. Be comfortable with the Beginner topics in this area first, then read the common pitfall carefully, since it is where most mistakes happen.
How long does it take to learn PCA?
Most learners can read the explanation and finish the interactive lesson in under an hour. Fluency comes from applying the idea, so plan a few short practice sessions with the quiz.
Which topics are related to PCA?
Dimensionality Reduction, Hyperparameter Tuning, Feature Engineering are closely connected. The full list is in the related topics section of this page.
What should I study after PCA?
Continue with Hyperparameter Tuning, the next topic in AI & Machine Learning.

Related topics