**Goal:** After completing this lab, you should be able to…

*Use*simulation to verify significance levels.*Use*simulation to estimate power.

In this lab we will use, but not focus on…

`R`

Markdown. This document will serve as a template. It is pre-formatted and already contains chunks that you need to complete.

Some additional notes:

- Please see
**Carmen**for information about submission, and grading. - You may use this document as a template. You do not need to remove directions. Chunks that require your input have a comment indicating to do so.

The following function will be used throughout this lab:

```
sim_p_value = function(mu = 0, sigma = 1, n = 25, mu_0 = 0) {
# sample from population
sample_data = rnorm(n = n, mean = mu, sd = sigma)
# calculate statistics
x_bar = mean(sample_data)
s = sd(sample_data)
t = (x_bar - mu_0) / (s / sqrt(n))
# calculate p-value
p_value = 2 * pt(-abs(t), df = n - 1)
# return p-value
p_value
}
```

This function simulates p-values for a two-sided, one sample \(t\)-test. That is, we assume that the data is sampled from a normal distribution, \(N(\mu, \sigma^2)\), and we are performing the test:

\[ H_0: \mu = \mu_0 \text{ vs } H_1: \mu \neq \mu_0 \]

The function takes as input four arguments:

`mu`

: The true value of \(\mu\) in the population`sigma`

: The true value of \(\sigma\) in the population`n`

: The sample size`mu_0`

: The hypothesized value of \(\mu\) which we call \(\mu_0\)

Let’s run the function and discuss what is happening.

```
set.seed(1)
sim_p_value(mu = 10, sigma = 3, n = 25, mu_0 = 13)
```

`## [1] 0.0002035751`

First, internally the function samples 25 observations from a normal distribution with a mean of 10 and a standard deviation of 3. Then, the function calculates and returns the p-value for testing

\[ H_0: \mu = 13 \text{ vs } H_1: \mu \neq 13 \]

Note that this is a “small” p-value, so with a significance level of say, \(\alpha = 0.05\) we would reject \(H_0\). (Which makes sense, the true value of \(\mu\) is not 13!)

Recall that \(\alpha\), the significance level is the probably of rejecting the null hypothesis when it is true.

\[ \alpha = P(\text{Reject } H_0 \mid H_0 \text{ True}) \]

Lets verify the significance level of the test

\[ H_0: \mu = 0 \text{ vs } H_1: \mu \neq 0 \]

The true distribution will be normal with a mean of 0 and a standard deviation of 1. We will use a sample size of 25. We will simulate this test 25000 times and use the simulated p-values to estimate the true \(\alpha\) of the test.

```
set.seed(42)
p_values = replicate(25000, sim_p_value(mu = 0, sigma = 1, n = 25, mu_0 = 0))
head(p_values)
```

`## [1] 0.4798228 0.1843076 0.4695465 0.7457961 0.4990247 0.3441031`

Here we use the `replicate()`

function to repeatedly run the `sim_p_value()`

function. This avoids needing to write a `for`

loop.

Notice that these p-values appear roughly uniform.

```
hist(p_values, probability = TRUE, col = "darkgrey",
main = "Histogram of P-Values, Null True", xlab = "p-values")
box()
grid()
```