Principal Component Analysis (PCA)

PCA is a statistical method that converts observations (data points) of possibly correlated variables into a smaller set of linearly uncorrelated variables that capture most of the information of the original data. Wikipedia: PCA

Key Statements
# Inputs: prepared_data, N_COMPONENTS

# Fit the model.
from sklearn import decomposition
model = decomposition.PCA(n_components=N_COMPONENTS)

# Get results.
pca_data = model.transform(prepared_data)
Working End-to-End Example
# Step 1: Import the libraries.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

import pandas as pd
from sklearn import decomposition

# Step 2: Set up the constants.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# We need to know how many components to make.

# There are some columns we won't use in this model.
# Note that if we plan to use the PCA components in a
# predictive model later, it's critical that we exclude
# the target feature here. With this data, we often
# predict the price, so we'll exclude that feature here.
FEATURES_TO_REMOVE = ['id', 'date', 'price']

# We need to know which features are categorical.
CATEGORICAL_FEATURES = ['waterfront', 'condition', 'zipcode']

# Step 3: Load in the raw data.
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# This assumes the data is in the same directory as this script.
# Here we load the data into a pandas DataFrame.
raw_data = pd.read_csv('kc_house_data.csv')

# It's helpful to take a quick look at the data.
print('Sample of loaded data:')

# Step 4: Set up the data.
# ~~~~~~~~~~~~~~~~~~~~~~~~

# Using drop() doesn't change raw_data, only the return value.
# The axis=1 keyword tells pandas to drop a column (not a row).
prepared_data = raw_data.drop(FEATURES_TO_REMOVE, axis=1)

# Turn categorical variables into dummy columns (0 or 1 values).
# Do this to avoid assuming a meaningful order of categories.
# Use drop_first to avoid multicollinearity among features.
prepared_data = pd.get_dummies(

# It's helpful to double check that the final data looks good.
print('Sample of data to use:')

# Step 5: Fit the model.
# ~~~~~~~~~~~~~~~~~~~~~~

model = decomposition.PCA(n_components=N_COMPONENTS)

# Yes, that's it!

# Step 6: Get the results.
# ~~~~~~~~~~~~~~~~~~~~~~~~

pca_data = pd.DataFrame(model.transform(prepared_data))


When PCA components are included in a predictive model, it's critical to exclude the target feature from those components.