Learning Objectives for Quiz 7

Inferential Statistics

1. State the three main tasks of inferential statistics.
2. State the three procedures used to perform the three main tasks of inferential statistics.
3. Explain why the outcomes of inferential procedures are themselves random.
4. Explain what it means for an inferential procedure to have desirable frequency properties.

Fisher’s Confidence Interval for \(\rho\)

1. State Fisher’s confidence interval for the population correlation \(\rho\) of a bivariate Gaussian based on his \(Z\)-transformation of the sample correlation \(R\).
2. Compute the hyperbolic functions \(\sinh x\), \(\cosh x\), and \(\tanh x\) and their inverses using R.
3. Compute Fisher’s confidence interval in R from two vectors x and y both:
   • directly from the formulas
   • using cor.test()

Hypothesis Tests

1. You are expected to know all of the intro stats-level material about hypothesis testing, including but not limited to:
   • setting up the null and alternative hypotheses for a claim
   • under a given pair of hypotheses, stating the meaning of a Type I or Type II Error
   • defining the Type I and Type II Error Rates of a test
   • defining the power of a test
   • performing a given hypothesis test using:
     1. a rejection region
     2. a confidence interval
     3. a \(P\)-value
2. Define the size of a hypothesis test.
3. Explain what it means for a hypothesis test to have (significance) level \(\alpha\).

Nonparametric Hypothesis Tests

1. Explain what it means for a distribution to be parametric.
2. Give examples of parametric distributions and state their parameters.
3. Explain the advantages of a nonparametric hypothesis test compared to a parametric hypothesis test.

The One-sample Sign Test

1. State the null and alternative hypotheses for the one-sample sign test.
2. State the test statistic for the one-sample sign test and its sampling distribution under the null hypothesis.
3. Give the rationale for the test statistic for the one-sample sign test.
4. Perform a one-sample sign test using SignTest() from the DescTools package.
5. Explain why there are a discrete set of \(P\)-values for the one-sample sign test.
6. Describe the general shape of the sampling distribution of \(P\)-values for a one-sample sign test when the null hypothesis is true.

Simulation

Big Picture

1. Explain the analogy between \(X_{1}, X_{2}, \ldots, X_{n} \stackrel{\text{iid}}{\sim} D\) (in math) and Xs <- rD(n) (in R).

Sampling Distributions via Simulation

1. Given a population distribution and a sample statistic that can be calculated using R, construct a single realization of the sample statistic in R.
2. Use replicate() to repeatedly generate a random sample and calculate an associated sample statistic.
3. Compare the empirical sampling distribution from simulation to a known sampling distribution from theory.

Coverage of Confidence Intervals via Simulation

1. Given a population distribution and an interval estimator that can be calculated using R, construct a single realization of the interval estimator, i.e. a confidence interval.
2. Use replicate() to repeatedly generate a random sample and calculate an associated confidence interval.
3. Check the coverage of confidence intervals returned by replicate() when given the true value of a population parameter.
4. Explain why the number of confidence intervals in a simulation that cover the true value follows a binomial distribution.
5. Use prop.test() to construct an interval estimate for the true coverage of a confidence interval using simulation.

Statistical Characteristics of Hypothesis Tests

1. Given a population distribution and its mean, simulate a one-sample \(t\)-test from that population under the null or alternative hypothesis using t.test().
2. Given a population distribution and its median, simulate a one-sample sign test from that population under the null or alternative hypothesis using SignTest() from the DescTools package.
3. Use replicate() to simulate many \(P\)-values for a one-sample \(t\)-test or sign test under the null or alternative hypotheses.
4. Use ecdf() to approximate either the Type I Error Rate or Power of a hypothesis test from the simulated \(P\)-values.
5. Explain what the sampling distribution of \(P\)-values should look like under a point null hypothesis \(H_{0} : \theta = \theta_{0}\) when the null hypothesis is true.
6. Explain what the sampling distribution of \(P\)-values should look like under a point null hypothesis \(H_{0} : \theta = \theta_{0}\) when the null hypothesis is false.

Parametric Distributions

1. Explain why the plug-in estimator for a property of a parametric distribution is so-named.
2. Given a parametric distribution, estimates for its parameters, and a function for how the parameters are related to a particular property of the distribution, compute a plug-in estimate for that property.

The Parametric Bootstrap

1. Explain the two “levels” of the parametric bootstrap in terms of what has actually happened in the real world and what is being simulated in bootstrap world.
2. State the “recipe” for generating bootstrap estimates from a parametric distribution.
3. Given a parametric distribution and estimators for its parameters, use the parametric bootstrap to obtain bootstrap estimates of a property of that distribution.

The Nonparametric Bootstrap

1. Compare and contrast the parametric and nonparametric bootstraps.
2. Explain the two “levels” of the nonparametric bootstrap in terms of what has actually happened in the real world and what is being simulated in bootstrap world.
3. State the “recipe” for generating bootstrap estimates using the nonparametric bootstrap.
4. Explain how to sample from the ECDF of a sample using sample().
5. Given an estimator for a property of a distribution, use the nonparametric bootstrap to obtain bootstrap estimates of that property of the distribution.
6. Explain under what scenarios a parametric or nonparametric bootstrap is more appropriate.

The Bootstrap Percentile Confidence Interval

1. Construct a coverage \(1 - \alpha\) bootstrap percentile confidence interval for a property of a distribution using either parametric or nonparametric bootstrap estimates of that property.