# Exercise 1

Let $$X_1, X_2, \ldots, X_n \stackrel{iid}{\sim} \text{Poisson}(\lambda)$$. That is

$f(x \mid \lambda) = \frac{\lambda^xe^{-\lambda}}{x!}, \quad x = 0, 1, 2, \ldots \ \ \lambda > 0$

(a) Obtain a method of moments estimator for $$\lambda$$, $$\tilde{\lambda}$$. Calculate an estimate using this estimator when

$x_{1} = 1, \ x_{2} = 2, \ x_{3} = 4, \ x_{4} = 2.$

(b) Find the maximum likelihood estimator for $$\lambda$$, $$\hat{\lambda}$$. Calculate an estimate using this estimator when

$x_{1} = 1, \ x_{2} = 2, \ x_{3} = 4, \ x_{4} = 2.$

(c) Find the maximum likelihood estimator of $$P[X = 4]$$, call it $$\hat{P}[X = 4]$$. Calculate an estimate using this estimator when

$x_{1} = 1, \ x_{2} = 2, \ x_{3} = 4, \ x_{4} = 2.$

# Exercise 2

Let $$X_1, X_2, \ldots, X_n \stackrel{iid}{\sim} N(\theta,\sigma^2)$$.

Find a method of moments estimator for the parameter vector $$\left(\theta, \sigma^2\right)$$.

# Exercise 3

Let $$X_1, X_2, \ldots, X_n \stackrel{iid}{\sim} N(1,\sigma^2)$$.

Find a method of moments estimator of $$\sigma^2$$, call it $$\tilde{\sigma}^2$$.

# Exercise 4

Let $$X_1, X_2, \ldots, X_n$$ be a random sample from a population with pdf

$f(x \mid \theta) = \frac{1}{\theta}x^{(1-\theta)/\theta}, \quad 0 < x < 1, \ 0 < \theta < \infty$

(a) Find the maximum likelihood estimator of $$\theta$$, call it $$\hat{\theta}$$. Calculate an estimate using this estimator when

$x_{1} = 0.10, \ x_{2} = 0.22, \ x_{3} = 0.54, \ x_{4} = 0.36.$

(b) Obtain a method of moments estimator for $$\theta$$, $$\tilde{\theta}$$. Calculate an estimate using this estimator when

$x_{1} = 0.10, \ x_{2} = 0.22, \ x_{3} = 0.54, \ x_{4} = 0.36.$

# Exercise 5

Let $$X_1, X_2, \ldots, X_n$$ iid from a population with pdf

$f(x \mid \theta) = \frac{\theta}{x^2}, \quad 0 < \theta \leq x$

Obtain the maximum likelihood estimator for $$\theta$$, $$\hat{\theta}$$.

# Exercise 6

Let $$X_{1}, X_{2}, \ldots X_{n}$$ be a random sample of size $$n$$ from a distribution with probability density function

$f(x, \alpha) = \alpha^{-2}xe^{-x/\alpha}, \quad x > 0, \ \alpha > 0$

(a) Obtain the maximum likelihood estimator of $$\alpha$$, $$\hat{\alpha}$$. Calculate the estimate when

$x_{1} = 0.25, \ x_{2} = 0.75, \ x_{3} = 1.50, \ x_{4} = 2.5, \ x_{5} = 2.0.$

(b) Obtain the method of moments estimator of $$\alpha$$, $$\tilde{\alpha}$$. Calculate the estimate when

$x_{1} = 0.25, \ x_{2} = 0.75, \ x_{3} = 1.50, \ x_{4} = 2.5, \ x_{5} = 2.0.$

Hint: Recall the probability density function of an exponential random variable.

$f(x \mid \theta) = \frac{1}{\theta}e^{-x/\theta}, \quad x > 0, \ \theta > 0$

Note that, the moments of this distribution are given by

$E[X^k] = \int_{0}^{\infty} \frac{x^k}{\theta}e^{-x/\theta} = k! \cdot \theta^k.$

This hint will also be useful in the next exercise.

# Exercise 7

Let $$X_{1}, X_{2}, \ldots X_{n}$$ be a random sample of size $$n$$ from a distribution with probability density function

$f(x \mid \beta) = \frac{1}{2 \beta^3} x^2 e^{-x/\beta}, \quad x > 0, \ \beta > 0$

(a) Obtain the maximum likelihood estimator of $$\beta$$, $$\hat{\beta}$$. Calculate the estimate when

$x_{1} = 2.00, \ x_{2} = 4.00, \ x_{3} = 7.50, \ x_{4} = 3.00.$

(b) Obtain the method of moments estimator of $$\beta$$, $$\tilde{\beta}$$. Calculate the estimate when

$x_{1} = 2.00, \ x_{2} = 4.00, \ x_{3} = 7.50, \ x_{4} = 3.00.$

# Exercise 8

Let $$Y_1, Y_2, \ldots, Y_n$$ be a random sample from a distribution with pdf

$f(y \mid \alpha) = \frac{2}{\alpha} \cdot y \cdot \exp\left\{-\frac{y^2}{\alpha}\right\}, \ y > 0, \ \alpha > 0.$

(a) Find the maximum likelihood estimator of $$\alpha$$.

(b) Let $$Z_1 = Y_1^2$$. Find the distribution of $$Z_1$$. Is the MLE for $$\alpha$$ an unbiased estimator of $$\alpha$$?

# Exercise 9

Let $$X$$ be a single observation from a $$\text{Binom}(n, p),$$ where $$p$$ is an unknown parameter. (In this case, we will consider $$n$$ known.)

(a) Find the maximum likelihood estimator (MLE) of $$p$$.

(b) Suppose you roll a 6-sided die 40 times and observe eight rolls of a 6. What is the maximum likelihood estimate of the probability of observing a 6?

(c) Using the same observed data, suppose you now plan to perform a second experiment with the same die, and will roll the die 5 more times. What is the maximum likelihood estimate of the probability that you will observe no 6’s in this next experiment?

# Exercise 10

Suppose that a random variable $$X$$ follows a discrete distribution, which is determined by a parameter $$\theta$$ which can take only two values, $$\theta = 1$$ or $$\theta = 2$$. The parameter $$\theta$$ is unknown.

• If $$\theta = 1$$, then $$X$$ follows a Poisson distribution with parameter $$\lambda = 2$$.
• If $$\theta = 2$$, then $$X$$ follows a Geometric distribution with parameter $$p = \frac{1}{4}$$.

Now suppose we observe $$X = 3$$. Based on this data, what is the maximum likelihood estimate of $$\theta$$?

# Exercise 11

Let $$Y_1, Y_2, \ldots, Y_n$$ be a random sample from a population with pdf

$f(y \mid \theta) = \dfrac{2\theta^2}{y^3}, \ \ \theta \le y < \infty$

Find the maximum likelihood estimator of $$\theta.$$.