Linear Regression and Maximum Likelihood
A Probabilistic View
In the previous lesson, we chose squared error as the objective for linear regression because errors can be positive or negative.
In this lesson, we look at linear regression from a probabilistic point of view. We'll see that the same squared error objective comes out of a statistical method called maximum likelihood estimation.
Maximum Likelihood Estimation
Maximum likelihood estimation (MLE) is a statistical method used in machine learning. It works by maximizing the likelihood function, which represents the probability of observing the data given specific parameter values.
Our dataset gives us both the inputs and the labeled outputs. So we try to maximize the likelihood of the actual output given the input.
Here is a small example. Suppose you want to estimate a coin's bias, so you flip it 100 times and get 60 heads.
MLE says the most likely probability of heads is , because that is the value that best explains your data.
The formula for the likelihood depends on the distribution the data comes from. Next, we'll apply this idea to linear regression.