• The fast spread of generative AI models has also led to the fast spread of opinions, predictions, and perspectives about them. Ultimately, the outcomes driven by emerging technologies are difficult to predict – especially outcomes related to employment and the marketplace. At the outset of the lesson, students might share strong opinions for / against the merits of AI and about if / when AI will impact employment rates. Encouraging these initial perspectives will help build interest in the lesson. However, we recommend quickly moving into the main part of the lesson, in which students use real data sets to empirically explore employment and AI. After this data-based analysis, the Discussion Question presents an excellent opportunity to then re-open the broader conversation about AI and the future of employment – now informed by the data analysis undertaken in the lesson.
• Throughout this lesson, it’s helpful to emphasize that the parameter of interest isn’t a proportion. Rather, it’s the difference between two proportions p₁ - p₂. Therefore, a parameter value of zero (p₁ - p₂ = 0) indicates no difference between population proportions. In other words, values close to zero indicate similarity between proportions. Continually pointing to the difference as the key quantity of interest will help students internalize this important framework for the rest of the unit.
• It’s also helpful to emphasize that the order of subtraction should remain consistent throughout the analysis. The interpretation of the observed difference, the expected difference, and the final conclusion all depend on using the same ordering for both the sample statistics and the population parameters.

First, download this lesson's Handout Key and read through its Discussion Question section. Then, check out our model discussion norms and the additional background notes below.

• Encourage students to consider both the evidence presented in the lesson and the limitations of that evidence. The data we analyze in the lesson describe changes in unemployment among recent computer science graduates, but they do not establish AI as the sole cause of those changes. The data also only come from recent years and, therefore, cannot establish a long-term trend on their own. These discussions will surface helpful themes from earlier in the course – e.g. generalizability, causal inference, and study design – that shape the scope of potential conclusions.
• Ultimately, it can be helpful to acknowledge that no one can predict the future – especially when it comes to the complex interactions between technology and the market. Therefore, this discussion will necessarily be at least partly speculative. Recognizing this fact will help students be curious and empathetic about one another’s views, and it will help them get comfortable with uncertainty – an important trait for any statistician.

• The data used in this lesson come from the American Community Survey (ACS), a stratified random sample of the US population conducted by the US Census Bureau each year. Because the ACS gathers detailed demographic, economic, and social data and uses rigorous sampling methods, it is widely used for research and public policy.
• Out of curiosity, we asked an AI chatbot (ChatGPT) what it thought about this lesson. After reviewing the lesson slides, here was its reply: “The central theme is timely and compelling because it uses students’ real concerns about AI and employment to motivate two-proportion inference, while the ATM comparison adds useful nuance and prevents the lesson from settling for an overly simple conclusion.” Stop, we’re blushing. (but also, see the unfortunate potential downsides of AI’s tendency towards flattery)

• The standard deviation of the sampling distribution for the difference between two proportions is introduced without derivation. The emphasis in this lesson is on interpreting the model and understanding when it is appropriate to apply it, rather than proving the formula itself. However, this optional video is available for students and instructors that would like to see its informal derivation.
• This lesson’s analysis uses the sample unemployment rate in 2013 (p̂₁ = 0.056) as the presumed true unemployment rate in both 2013 and 2024. A more precise analysis would utilize the pooled or combined sample unemployment rate from both years \( (\hat{p}_c = \frac{13+121}{233+867} = \frac{134}{1100} \approx 12.2\%) \) as the presumed true unemployment rate for both years. For now, we wanted to provide a simpler method, so that students wouldn’t get bogged down in these calculations and lose sight of the overall intuition. Instead, students will encounter the pooled or combined proportion when they learn about the two-sample z-test for the difference between population proportions in Lesson 3.B.3. We believe that is a more appropriate place to introduce this method, after students have developed intuition for the overall approach. That said, with either analysis, students will arrive at similar results and find convincing evidence that the true unemployment rate among recent computer science graduates is higher in 2024.