Lesson 2.1 - Types of Studies

Key Question: How do we know if vaccines actually work?

Content: Investigative Questions | Obervational vs Experimental | Confounding | Generalizability

Video

Student Items

Handout: pdf, doc

Mastery Check: link

Teacher Items

Handout Key: pdf, doc

Mastery Check Key: link

Slide Deck: pdf, ppt

Course Resources

Resources for teaching our High School Statistics curriculum.

  • Lesson Flow - timing and flow of class, using our lesson materials
  • Pacing Guide - pacing our units, with daily or block schedules
  • Alignment Guide - aligning our lessons to national and state standards for high school statistics
  • Classroom Routines - a guidebook of classroom routines embedded within our lessons

Teaching Resources

Resources for teaching with Skew The Script.

Lesson Notes

Lesson-specific insights from the creators of this lesson.

GIF

In this lesson, students explore how evidence becomes convincing. Specifically, they investigate how study design affects the strength and scope of conclusions. Then, using data from the Covid-19 pandemic, students answer an important question: How do we know if vaccines actually work?

Learning Targets
  • Compose effective investigative questions
  • Determine if a study is observational or experimental
  • Describe confounding variables and determine if causal conclusions are possible
  • Determine the generalizability of a study from its sampling method
Learning Progression

This lesson serves as an overview for the remainder of the unit, featuring the most important ideas from sampling and experiments. As such, it introduces several new concepts and quite a few vocabulary terms. Many of the ideas that students encounter, including investigative questions, confounding variables, sampling design, experimental design, and generalizability, will be addressed again in later lessons. At this stage, as students are beginning to use the new language introduced, the general intuition and framing is more important than mastering each individual concept.


Before proceeding: Familiarize yourself with the lesson materials linked above (e.g. handout, handout key, slides, video). Then, for additional background and teaching tips from the lesson creators, check out the sections below.


  • Students look at real data related to Covid-19 and the flu in this lesson. This context could be difficult for some students, as they may have been impacted by loss during the pandemic. At the same time, these students may also find this lesson context – and the scientific breakthroughs it portrays – to be deeply motivating. Approaching this lesson with seriousness, gentleness, and care, along with a focus on the data and statistical reasoning, can help create an environment in which all students feel motivated and ready to engage. That said, it can also be helpful to have counseling or other supports at the ready for students with personal connections to the lesson context. Being on the lookout for students deeply affected by the topic and offering options for those moved by the discussion will assure that students continue to develop their sense of belonging during this lesson and for other topics they care deeply about.
  • At the same time, due to their age at the time of the Covid-19 pandemic, some current high school students may have limited awareness of the severity of the disease when it first broke out, before humanity had built up more resistance to it. It’s important to ground the lesson in the numbers and video linked in the first slides of the slide deck, which give a very real sense of the initial severity and toll of the disease.
  • It’s important to emphasize that the first Covid-19 vaccines were created, tested, and approved in less than a year – a rapid timeline that broke records. For many, this was seen as a scientific breakthrough. However, for others, the rapid timeline created skepticism. In that short timeline, how did scientists establish that the vaccines actually worked? That’s what we explore in the lesson.
  • This lesson addresses vaccine efficacy. Another important question that students might bring up is vaccine safety. When subjects are randomly assigned a new vaccine or a placebo, researchers monitor for severe side effects in the vaccine group. In the Moderna vaccine trial, they found that “serious adverse events were rare, and the incidence was similar in the two groups.” Hence, they concluded that “aside from transient local and systemic reactions, no safety concerns were identified.”

First, download this lesson's slide deck and handout key to see the prompt and sample responses for the Lesson Starter. Then, check out the additional background notes below.

Instructional routine: Notice & Wonder. The lesson handout provides a Notice & Wonder T-frame for students to capture their notes and ideas. It is important that students recognize the difference between a noticing (observation) and a wondering (question that comes to mind). You can find more background on implementing a Notice & Wonder here.

Purpose & Background: The goal of this Lesson Starter is to open the unit with a striking graph that will relate to a confounding variable later in the lesson. Purposely shown without title or axis labels, the focus of the Lesson Starter is to consider the three curves without complete information. Even if students recognize the timeline as that of the pandemic, they won’t know what was being tracked (illness, death, testing, etc.) in this data, so encourage them to notice and wonder in ways specific to the graph, rather than a presumed context. Ultimately, their observations of the patterns will make their later consideration of the graph in the lesson (this time with labels and in context) even more powerful. These data show morbidity rates per 100,000 people, which will become apparent later in the lesson.

First, download this lesson's handout key and read through its Discussion Question section. Then, check out our model discussion norms and the additional background notes below.

  • In the original Moderna vaccine trial among adults, the estimated vaccine efficacy was 94.1%. In a follow-up trial among children, the vaccine was found to be 36.8% effective for 2-to-5-year-olds and 50.6% effective for 6-to-23-month-olds. Therefore, the efficacy was substantially higher among adults than among young children. This provides an excellent teaching example of how the study results (94.1% effectiveness) only fully generalize to people similar to those studied in the original trial (adults).
  • Even though the vaccine efficacy among children is lower, it’s still not 0, so the vaccine still provides some level of protection for children. For extra background, here’s a lesson from the New York Times that shows the math behind calculating vaccine efficacy.
  • The CDC has several ways of estimating the death toll from respiratory diseases. In the lesson, we use the most conservative CDC figures, which come from counts of official patient death certificates. This likely provides an underestimate of the true number of flu and covid-related deaths, which are likely underreported on death certificates.
  • The studies for each major Covid-19 vaccine can be found here (linked): Moderna, Pfizer, and Johnson & Johnson.
  • Students may need further explanation of asymptomatic, symptomatic, and severe cases. Asymptomatic cases are ones in which the infected individual does not feel sick; yet, they’re still infected and can spread the disease. In symptomatic cases, the individual feels sick, but they’re not in critical condition. In severe cases, the individual is in critical condition.
  • The main trial discussed in this lesson – the Moderna vaccine trial – found that “aside from transient local and systemic reactions, no safety concerns were identified.” However, patients were monitored for a limited amount of time. What about potential side effects that could emerge 10, 20, or 30 years after taking the vaccine? According to immunology experts, in the history of vaccines, no vaccines have ever produced long-term side effects that only emerge years after injection. Instead, all side effects (including rare ones) emerge within 6-8 weeks of injection. Further, the body quickly breaks down the mRNA strands introduced by the Moderna vaccine within a few hours or days after injection.
  • Common synonyms for the explanatory variable include factor, independent variable, predictor, feature, or input. Common synonyms for the response variable include dependent variable, target, or output.
  • The main outcome measured in vaccine trials is vaccine efficacy. For extra background, here’s a lesson from the New York Times that shows the math behind calculating vaccine efficacy.
  • It’s important to emphasize that generalizability requires not just random sampling, but also sampling from the correct target population. If we randomly sample from the wrong population or from just a subset of the target population, we cannot generalize our results to the whole target population.

Student Supports

Lesson-specific resources to support all learners.

  • For confounding, it can be helpful to provide a simpler example, before jumping to the more complex health-based example in the lesson. For instance, the ice cream example from Ionica Smeets.
  • Similarly, for motivating random sampling, providing a simpler example before the health-based example can be helpful. For instance, imagine a school principal is considering moving the school day back an hour, such that first period starts an hour later and school ends an hour later. She decides to survey the first 100 students who arrive on campus for their opinion about this change. Why might the opinion of her sample not be representative of the whole study body?
  • Mathematical Language Routines useful for this lesson: Collect and Display (MLR2) – In this language rich unit, this lesson is a good place to begin a visual reference for the vocabulary students will see. As students progress through this lesson and those that follow, capture their language and strategies to support understanding and use of formal terminology to explain their processes and thinking. Add terms and descriptors to this collection throughout the unit, when applicable.
  • Vocabulary used in the context of the lesson may include words that are unfamiliar or have several meanings. In particular, the following mathematical terms may need clarification or a definition provided:
    • Explanatory variable
    • Response variable
    • Factor
    • Outcome
    • Causation
    • Association
    • Observational study
    • Confounding variable
    • Survey
    • Experiment
    • Census
    • Simple Random Sample (SRS)
    • Inference
    • Generalizability
  • In addition, the following contextual terms may need clarification or a definition provided:
    • Vaccine
    • Placebo
    • Investigative
    • Imposed
    • Contagious
    • Asymptomatic
    • Symptomatic
    • Severe illness
  • Common synonyms
    • Explanatory variable may also be referred to as factor, independent variable, predictor, feature, or input.
    • Response variable may also be referred to as dependent variable, target, or output.