Lesson 2.6 - Informed Consent & Natural Experiments
Key Question: Does college have a causal impact on earnings?
Content: Informed Consent | Natural Experiments
Video
Course Resources
Resources for teaching our High School Statistics curriculum.
- Lesson Flow - timing and flow of class, using our lesson materials
- Pacing Guide - pacing our units, with daily or block schedules
- Alignment Guide - aligning our lessons to national and state standards for high school statistics
- Classroom Routines - a guidebook of classroom routines embedded within our lessons
Teaching Resources
Resources for teaching with Skew The Script.
- Discussion Norms - our model discussion norms for the classroom
- Letter to Parents - letter to share with parents about our nonpartisan approach
- Teaching Math on Civic Topics - tips for teaching math lessons that cover civic topics
Lesson Notes
Lesson-specific insights from the creators of this lesson.
College graduates tend to make more money than high school graduates. However, does obtaining a college degree have a causal relationship with higher wages? Or do colleges tend to attract skilled and motivated people, who would earn high wages anyways? In this lesson, students tackle this question. Along the way, they find that an experiment in this context is impossible, due to the inability to obtain informed consent. So, they explore a different way to address this causal question: a natural experiment.
- Recognize practical and ethical considerations of informed consent among human subjects
- Identify natural experiments and describe the scope of their conclusions
Considerations about personally identifiable information connect directly to the first topic of this lesson: informed consent. In addition, as students explore the unique aspects of natural experiments, they have the opportunity to surface and solidify their understanding of many of the unit’s key concepts. As the unit concludes, students should be able to distinguish between experiments, natural experiments and observational studies, recognizing their differences in terms of causal-and-effect conclusions and generalizability.
Before proceeding: Familiarize yourself with the lesson materials linked above (e.g. handout, handout key, slides, video). Then, for additional background and teaching tips from the lesson creators, check out the sections below.
- The informed consent activity should take place after the Lesson Synthesis and before the Practice sections of the lesson. The number of questions in the Practice section of the lesson is smaller than usual, which creates additional class time for the activity.
- The informed consent activity handout includes reflection questions and 4 versions of informed consent forms. Each student should receive 1 of the 4 versions and a copy of all the reflection questions.
- This activity is run similarly to a jigsaw protocol. First, have students gather together into “home groups,” with 4 students per home group. These groups can be self-selected, randomly assigned, or teacher-assigned. In each home group, give each of the students one of the 4 informed consent form versions – such that each student in the group gets a different version. Then, give students ample time to individually review the form and draft their own answers to the reflection questions.
- After independently answering the questions, students will transition into “expert groups” – groups in which every student has reviewed the same informed consent form version. Because there are 4 versions of the form, instructors can assign each of the 4 corners of the classroom to a certain version, directing students to gather in the assigned corner. In expert groups, students will discuss their answers to the reflection questions together, to flesh out their thinking before returning to their home group.
- Each expert group should be able to identify at least one way that their version of the informed consent form doesn’t meet the full definition of informed consent. This is intentional and should support the whole group discussion later in the activity. See the activity key for sample responses for all of the reflection questions.
- If there are more than 20 students in class, expert groups may be too large to ensure that all students participate. So, creating parallel groups (e.g. two expert groups for each form) will encourage all students to contribute.
- Finally, students return to their home groups. In home groups, each student shares their findings and answers to the reflection questions about their “expert” form version. After this sharing in home groups, the instructor facilitates a whole group discussion about all 4 form versions.
- Since this is the last lesson of the unit, during the Lesson Starter, the Informed Consent Activity, and the Discussion Question, take available opportunities to bring up previous learning about types of studies, sampling methods, bias, and experimental design, in order to formatively assess students’ progress and identify areas for support or review.
- There are ample opportunities for students to relate the concept of informed consent to signing up for games, apps, and services online. While these connections to students’ daily lives are helpful, they should also be connected back to the focus of the lesson: informed consent for research.
- As students review the components of informed consent, it will be helpful to resurface the personally identifiable information (PII) categories from Lesson 2.5 and identify those at risk in the various consent forms of the Informed Consent Activity.
- The consent forms in the activity are fictitious. However, if a local institution (e.g. a nearby University or lab) is willing to provide reproducible versions of their consent forms, students may find additional connections with research happening in the region.
First, download this lesson's slide deck and handout key to see the prompt and sample responses for the Lesson Starter. Then, check out the additional background notes below.
Instructional routine: Ten-Minute Talk. The lesson provides space for students to jot down their thoughts on the prompt before discussing with a partner and engaging in whole group discussion. You can find more background on implementing a Ten-Minute talk here.
Purpose & Background: The goal of this Lesson Starter is to engage students in thinking about absolute boundaries or cut-offs, and whether individuals who are close to a cutoff should be excluded. While this discussion is likely to hone in on ‘fairness,’ it informally introduces students to thinking about individuals ‘near the margins’ – which is the central concept behind the natural experiment introduced in the lesson (the effect of college, for students near a GPA cutoff). Discussion about how height limits could cut off amusement park guests’ opportunity for a fun ride may also contribute to later consideration of how admission limits may cut off opportunities from prospective students.
First, download this lesson's handout key and read through its Discussion Question section. Then, check out our model discussion norms and the additional background notes below.
- Students may veer into unproductive discussion of applicants’ “innate ability.” Some extra framing:
- Imagine that two applicants – applicant A and B – have the same innate ability. Applicant A attended a higher-performing high school. Applicant B attended a lower-performing school. Due to this inequity, applicant B is applying with a lower GPA and lower test scores.
- In theory, Applicant B should perform just as well as Applicant A if they are in the same school setting for college. However, if the academic gap created by attending a worse high school is too severe, it may prevent Applicant B from being successful.
- What do the results of our study suggest about the above scenario if Applicant B is just above the admission margin? Just below the admission margin? Far below the margin?
- Encourage students to think about this problem in terms of what was studied: the outcomes of admitted students. Students may have ideas about how lowering a GPA cutoff might affect University academics, University alumni relations, and other factors. However, we don’t have any evidence that speaks to these considerations.
- Additional ideas that could be discussed: What data was collected for the Florida College Study? Which classifies as personally identifiable? What would the participants need to give permission for?
- In the introduction, we identify high motivation and skill as possible confounding factors that may imply correlation rather than causation. However, as the New York Federal Reserve notes: “a number of studies that attempt to correct for this possibility find a similar return. In part, this is likely because in today’s labor market a college degree continues to serve as a gateway to professional occupations that offer better opportunities for wage growth over the life cycle.” This is further evidence, beyond the natural experiment explored in the lesson, that college may have a causal impact on wages.
- In addition to regression discontinuity (covered in this lesson), there are other types of natural experiments and quasi-experiments, including instrumental variable analysis and difference-in-differences. For the purpose of this course, students do not need to know the particulars of any quasi-experimental method. Instead, the primary focus is understanding that certain situations can arise in observational data that create “as good as randomly assigned” groups. When these scenarios meet all assumptions, they can be analyzed for causal inference.
- Natural experiments have broad applications in fields where random assignment to treatment isn’t commonplace, such as in education. In fact, researchers used a difference-in-differences natural experiment to determine that Skew The Script had a positive causal impact on AP Statistics test scores in Texas.
Student Supports
Lesson-specific resources to support all learners.
- Vocabulary used in the context of the lesson may include words that are unfamiliar or have several meanings. In particular, the following mathematical terms may need clarification or a definition provided:
- Regression discontinuity
- In addition, the following contextual terms may need clarification or a definition provided:
- Informed consent
- Rate of return