Week 9

This week, we will evaluate the goodness of point estimators.
Published

October 16, 2022

Learning Outcomes

Monday

  • Sufficiency

Wednesday

  • MVUE
  • Rao-Blackwell Theorem
  • MLE Properties

Reading

Day Reading
Monday’s Lecture MMS: 7.1,7.3,7.4
Wednesday’s Lecture MMS: 7.4

Homework

HW 4 can be found here. It is due October 21 at 11:59 PM. HW 5 will be posted soon.

Important Concepts

Sufficiency

Sufficiency evaluates whether a statistic (or estimator) contains enough information of a parameter \(\theta\). In essence a statistic is considered sufficient to infer \(\theta\) if it provides enough information about \(\theta\).

Let \(X_1,\ldots,X_n\) be a random sample from a distribution with parameter \(\theta\). A statistic \(T=t(X_1,\ldots,X_n)\) is said to be sufficient for making inferences of a parameter \(\theta\) if condition joint distribution of \(X_1,\ldots,X_n\) given \(T=t\) does not depend on \(\theta\).

Factorization Theorem

Let \(f(x_1, \ldots, x_n;\theta)\) be the joint PMF of PDF of \(X_1, \ldots,X_n\). Then \(T=t(X_1,\ldots,X_n)\) is a sufficient statistic for \(\theta\) if and only if there exist 2 nonnegative functions, \(g\) and \(h\), such that

\[ f(x_1,\ldots,x_n) = g\{t(x_1,\ldots,x_n);\theta\}h(x_1,\ldots,x_n). \]

Minimum Sufficient Statistics

A minimum sufficient statistic is a sufficient statistic that has the smallest dimensionality, which represents the greatest possible reduction of the data without any information loss.

Rao-Blackwell Theorem

Let the joint distribution of \(X_1,\ldots,X_n\) depend on parameter \(\theta\) and let \(T\) be a sufficient statistic for \(\theta\). If we consider estimating \(h(\theta)\), a function of \(\theta\). If \(U\) is an unbiased estimator of \(h(\theta)\), then the estimator \(U^*=E(U|T)\) is also an unbiased estimator of \(h(\theta)\) and has a variance no larger than \(U\).

Minimum Variance Unbiased Estimator

Let \(\mathcal U\) be a set of all unbiased estimators \(T\) of \(\theta\in\Theta\) and \(E(T^2)<\infty\). An estimator \(T_0\) is said to be the minimum variance unbiased estimator for \(\theta\) if

\[ E\{(T_0-\theta)^2\}\le E\{(T-\theta)^2\} \]

for all \(\theta\in\Theta\) and every \(T\in \mathcal U\).

Resources

You must log on to your CI Google account to access the video.

Press the “b” key to access hidden notes.

Lecture Slides Videos
Monday Slides Video
Wednesday Slides Video