Lesson 3.A.5 - Confidence & Margins of Error
Key Question: How did researchers expose the Flint Water Crisis?
Content: Critical Values | Sample Size Calculations
Alignment: CED Topic 3.3.D-3.4
Video
Course Resources
Resources for teaching our AP® Statistics curriculum.
- Lesson Flow - timing and flow of class, using our lesson materials
- Pacing Guide - pacing our units, with daily or block schedules
- CED Alignment Guide - aligning our lessons to the AP® Statistics Course and Exam Description
Teaching Resources
Resources for teaching with Skew The Script.
- Discussion Norms - our model discussion norms for the classroom
- Letter to Parents - letter to share with parents about our nonpartisan approach
- Teaching Math on Civic Topics - tips for teaching math lessons that cover civic topics
Lesson Notes
Lesson-specific insights from the creators of this lesson.
In 2014, officials in Flint, Michigan changed the city's water source from the Detroit water system to the local Flint River. The move cut costs for the city. However, residents began noticing a change in their water’s color and taste. Then, they started experiencing health effects. With officials insisting the water was fine, a team of citizens and scientists gathered their own samples from the water system to make an independent determination. In this lesson, students explore how their data and methods exposed the water crisis and brought change for the city.
- Determine critical values for z-intervals
- Calculate and interpret a one-sample z-interval for a population proportion (at all confidence levels)
- Determine how changes in confidence levels and sample size affect interval width
- Calculate the sample size needed to guarantee a certain margin of error
Before proceeding: Familiarize yourself with the lesson materials linked above (e.g. handout, handout key, slides, video). Then, for additional background and teaching tips from the lesson creators, check out the sections below.
- To provide additional historical context for the Flint Water Crisis, briefly explain that concerns about water quality were initially dismissed by officials, and that outside researchers used statistical sampling and confidence intervals to provide evidence that the city’s water system was unsafe. This reinforces the broader idea that statistical inference can be used to investigate public claims and shed light on real issues.
- This lesson builds on the previous lesson about confidence intervals for proportions, shifting the focus toward understanding how confidence levels and sample sizes affect margins of error and interval width. Encourage students to move beyond viewing confidence intervals as procedures and toward understanding the underlying tradeoffs between confidence, variability, and precision.
- Higher confidence levels require larger critical values, which produce wider intervals. Lower confidence levels produce narrower intervals, but with less certainty that the interval captures the true population value. Emphasizing this tradeoff with students can be helpful, as the concept will continue to be important throughout later inference topics.
First, download this lesson's Handout Key and read through its Discussion Question section. Then, check out our model discussion norms and the additional background notes below.
- Students may choose between several approaches to calculate sample size in this lesson. Some students may use a version of the margin of error formula already solved for n, while others may solve the inequality algebraically from the original formula. Some students may also use a more procedural “plug-and-chug” approach with calculator support. All three approaches are mathematically valid and comparing them can help students recognize the underlying structure of the calculation rather than viewing it as a memorized procedure.
- Students may wonder why sample size calculations commonly use p̂ = 0.5. The quantity p̂(1 − p̂) reaches its maximum possible value when p̂ = 0.5, which produces the largest possible standard error and therefore the widest possible interval. Using p̂ = 0.5 creates the most conservative sample size estimate because it guarantees the desired margin of error regardless of the true population proportion.
- This lesson continues the Flint Water Crisis context from the previous lesson but now shifts attention toward the precision of estimates rather than simply constructing a confidence interval. The comparison between the mock study and the larger Virginia Tech study provides a natural opportunity to discuss how increased sample size reduces variability and produces narrower intervals.
- The EPA regulation discussed in the lesson states that no more than 10% of homes should exceed the lead threshold. Students sometimes interpret this as meaning zero lead in the water in the rest of the homes. Clarify that the regulation concerns the proportion of homes above a specified cutoff level, rather than whether any lead is present at all.
- Students may initially believe that a narrower interval automatically provides “better” evidence. This lesson helps refine that idea by showing that interval width depends on both confidence level and sample size. Narrower intervals obtained by lowering the confidence level come at the cost of reduced confidence in the method’s long-run success rate.
- The relationship between higher sample sizes and lower spread connects directly to the law of large numbers introduced in lesson 2.A.2. For example, if we flip a fair coin only twice, there is a sizable chance that 100% of the flips will be Heads. If we flip the coin 1,000 times, getting 100% Heads becomes extraordinarily unlikely, and the sample proportion will likely be very close to the true probability of 50%. This same idea applies to sampling distributions: as sample size increases, estimates become more tightly clustered around the true population value, producing more precise intervals.
- Distinguishing between “evidence” and “convincing evidence” can be a subtle task. A sample proportion above the EPA’s 10% threshold is certainly evidence that the true population proportion may also exceed 10%, but one sample alone may not be convincing because different random samples can produce different results. Confidence intervals help students account for this uncertainty by showing a range of plausible values for the population proportion. This lesson helps students move from simply noticing sample results to viewing those results in the broader context of sampling variability.
- Students sometimes wonder why the critical value is written as z* rather than simply z. A useful distinction is that a z-score typically refers to a calculated standardized value, which may be positive or negative. In contrast, z* is always positive and refers to the specific value chosen to capture a desired level of confidence. Conceptually, z* represents the number of standard errors the interval extends away from the point estimate in each direction.
- Students may choose to memorize common critical values such as z* = 1.645 for 90% confidence, z* = 1.96 for 95% confidence, and z* = 2.576 for 99% confidence. However, it is still important for students to understand how these values are obtained using inverse normal calculations. Encourage students to sketch the normal curve, label tail areas, and connect the confidence level to the central area of the distribution before using technology to calculate the critical value.
Student Supports
Lesson-specific resources to support all learners.
- To support students in understanding the relationship between sample size and precision, it can help to surface the proportional reasoning that dividing by a larger number produces a smaller result, while dividing by a smaller number produces a larger result. Since sample size appears in the denominator of the standard error formula, acting as the divisor in these calculations, larger samples reduce variability in the sampling distribution.
- Reinforce the importance of consistently using the 5C Method when solving inference problems. Even when students become comfortable with calculations, the organizational structure helps ensure that they communicate all required components clearly and completely on AP Exam free-response questions.
- Emphasize that sample sizes must always be whole numbers. When sample size calculations produce a decimal value, students should always round up rather than round down. Rounding down would produce a sample that is slightly too small to guarantee the desired margin of error, while rounding up ensures the requirement is fully satisfied.
- Vocabulary used in the context of the lesson may include words that are unfamiliar or have several meanings. In particular, the following mathematical terms may need clarification or a definition provided:
- Confidence interval
- Confidence level
- Margin of error
- Sample size
- In addition, the following contextual terms may need clarification or a definition provided:
- Environmental Protection Agency (EPA)
- Parts per billion (ppb, in reference to measured lead levels in water)
- To support students in distinguishing the term “confidence interval” from “confidence level,” encourage them to separately identify the interval endpoints, the center, the margin of error, and the confidence level before interpreting the interval in context.