Week 7

This week, we will learn about Maximum Likelihood Estimators and Method of Moments Estimators.

Learning Outcomes

Monday

• Maximum Likelihood Approach

Wednesday

• Method of Moments

Reading

Day	Reading
Monday’s Lecture	MMS: 7.2
Wednesday’s Lecture	MMS: 7.2

Homework

HW 4 can be found here. It is due October 14 at 11:59 PM.

Important Concepts

Data

Let $X_1,\dots ,X_n\stackrel{iid}{\sim }F(\mathit{\theta })$ where $F(\cdot )$ is a known distribution function and $\mathit{\theta }$ is a vector of parameters. Let $\mathit{X}=(X_1,\dots ,X_n{)}^{\mathrm{T}}$, be the sample collected.

Maximum Likelihood Estimator

Likelihood Function

Using the joint pdf or pmf of the sample $\mathit{X}$, the likelihood function is a function of $\mathit{\theta }$, given the observed data $\mathit{X}=\mathit{x}$, defined as

$$L(\mathit{\theta }|\mathit{x})=f(\mathit{x}|\mathit{\theta })$$

If the data is iid, then

$$f(\mathit{x}|\mathit{\theta })=\prod _{i=1}^{n}f(x_i|\mathit{\theta })$$

Estimator

The maximum likelihood estimator are the estimates of $\mathit{\theta }$ that maximize $L(\mathit{\theta })$.

Log-Likelihood Approach

If $\ln \{L(\mathit{\theta })\}$ is monotone of $\mathit{\theta }$, then maximizing $\ln \{L(\mathit{\theta })\}$ will yield the maximum likelihood estimators.

Method of Moments

Let the $k$th moment be defined as ${\mu }_k$ and the corresponding $k$th moment average $\frac{1}{n}\sum _{i=1}^n{X}_i^k$:

$${\mu }_k=\frac{1}{n}\sum _{i=1}^n{X}_i^k.$$

The parameter estimates are for $t$ parameters are the solutions for ${\mu }_k$ for $k=1,\dots ,t$.

Resources

You must log on to your CI Google account to access the video.

Press the “b” key to access hidden notes.

Lecture	Slides	Videos
Monday	Slides	N/A
Wednesday	Slides

Extra Resources on Maximum Likelihood Estimators

Probability 4 Data Science

MLE Basics

MLE of Poisson

MLE of Uniform

MLE of Binomial

In MMS Chapter 7:

Problems: 27.b; 33.a

Extra Resources on Method of Moments Estimators

Probability 4 Data Science

Method of Moments Basics

Methods of Moments: Mean and Variance