Generalized Linear Model with Principal Components

4 min readSep 29, 2020

This article is an overview of the mathematics behind the Generalized Linear Model (GLM) and Principal Components. I will break the whole article in various segments of Questions which I generally ask myself before implementing any method.

The general form of the model:

Geometrical view and projection matrices

It tells us that the column space of X spans 2 dimensions.

What if we have lakhs of dimension then how would it look like? Well somehow similar to a dense graph like this-

PCA creates a linear transformation that transforms the data points to a new coordinate system. Based on the number of features or column we can calculate the number of principal components,

The data which has the greatest variance lie on the 1st axis (i.e. 1st principal component).

Components of GLM

Steps to fit a model

Describe the model→ glm()
Fit the model→ .fit()
Summarize the model→ .summary()
Make model predictions→ .predict()

Two lines to fit a GLM model,

Example-

import statsmodels.api as sm
from statsmodels.formula.api import glm
model = glm(formula = ‘y ~ X’ , data = df, family = sm.families.).fit

Sample case-

Let's consider these COVID cases. The person who has lower immunity they have a high risk to get the disease.

For example- In a hospital Doctors thought to give the shots to OLD people (age > 50). They randomly selected 50 people for cross-checking, if they got the shot or not?

Here outcome Yi is a 0–1 random variable, right? 0(they did not get shot) 1(got shot)

i.e. µi = Z(Yi) = πi

Variance function, Var(Yi) = πi(1 − πi) = µi(1 − µi)

To model the dependence of Yi on covariates Xi1, . . . , Xi,p−1 is by usingthe logit transform

logit(πi) = β0 +(Summation over j=1 to p-1)βjXij

**Logit transform → Inverse → Derivative**

Link function Transforms the expected value of y and Enables linear combinations Many techniques from linear models apply to GLMs.

Family Argument

Sample Model Summary

model.summary()

Principal Components

Before discussing principal components we know that PCA is a useful tool to describe variability patterns in the data. One of the goals of PCA is to reduce a set of ‘r’ variables into a meaningful subset of variables which are linear combinations of the original ones, although there will be as many principal components created as the number of original variables.

In principle components analysis:

→ Eigenvalues indicate the magnitude of variances of the (PC’s)

→ Eigenvectors indicate direction of the PC’s.