Week 7

This week, we will learn about Maximum Likelihood Estimators and Method of Moments Estimators.
Published

October 3, 2022

Learning Outcomes

Monday

  • Maximum Likelihood Approach

Wednesday

  • Method of Moments

Reading

Day Reading
Monday’s Lecture MMS: 7.2
Wednesday’s Lecture MMS: 7.2

Homework

HW 4 can be found here. It is due October 14 at 11:59 PM.

Important Concepts

Data

Let X1,,XniidF(θ) where F() is a known distribution function and θ is a vector of parameters. Let X=(X1,,Xn)T, be the sample collected.

Maximum Likelihood Estimator

Likelihood Function

Using the joint pdf or pmf of the sample X, the likelihood function is a function of θ, given the observed data X=x, defined as

L(θ|x)=f(x|θ)

If the data is iid, then

f(x|θ)=i=1nf(xi|θ)

Estimator

The maximum likelihood estimator are the estimates of θ that maximize L(θ).

Log-Likelihood Approach

If ln{L(θ)} is monotone of θ, then maximizing ln{L(θ)} will yield the maximum likelihood estimators.

Method of Moments

Let the kth moment be defined as μk and the corresponding kth moment average 1ni=1nXik:

μk=1ni=1nXik.

The parameter estimates are for t parameters are the solutions for μk for k=1,,t.

Resources

You must log on to your CI Google account to access the video.

Press the “b” key to access hidden notes.

Lecture Slides Videos
Monday Slides N/A
Wednesday Slides

Extra Resources on Maximum Likelihood Estimators

Probability 4 Data Science

MLE Basics

MLE of Poisson

MLE of Uniform

MLE of Binomial

In MMS Chapter 7:

Problems: 27.b; 33.a

Extra Resources on Method of Moments Estimators

Probability 4 Data Science

Method of Moments Basics

Methods of Moments: Mean and Variance