This applet examines the overlap fallacy. Two 95% confidence intervals can overlap and still have a p-value that suggests that the difference is statistically significant at a p-value less than 0.10 for a two-sample t test.
Type of Material:
Java applet that can be used individually by a student or as a demonstration by the instructor. Students probably will find the applet confusing without some guidance from instructor.
Students will need to have this applet explained to them prior to use on their own. Instructors should plan to spend 5-10 minutes in-class demonstrating the applet if the instructor expects to assign the applet for further explanation "on your own."
Would work very well in a lab setting where instructor demonstrates, students explore, instructor brings students back together for wrap-up.
Java required to run, but works with Firefox, Chrome, and IE (not tested on Safari).
Identify Major Learning Goals:
1. Understand that individual 95% confidence intervals can overlap but a significance test for the difference between two means may have a significance level below 0.05
2. Emphasize relation between overlapping confidence intervals and hypothesis test for difference between two means.
Target Student Population:
Sophisticated introductory statistics students AP Statistics and beyond.
Links to articles demonstrating the overlap fallacy are appropriate for first level mathematical stats students.
Prerequisite Knowledge or Skills:
Students should understand confidence interval estimation and hypothesis testing for two samples before this applet/game.
Evaluation and Observation
This applet addresses a very common misinterpretation about confidence intervals and/or SE bars for individual means. This is relevant to students who are preparing for advanced studies in fields where quantitative research is common. The guide is very useful and necessary. The motivation by pictures at the end is good.
The instructor needs to introduce the material and should expect to spend approximately 15 minutes investigating this applet prior to class.
The game asks to user to estimate where the CI’s would overlap or not based upon different p-value outcomes of a proper two-sample t test. There is actually an easy formula for computing the probability of overlap (the article is cited in the handout). But this can be visually difficult to do and the transfer of this skill to a research article where the CI’s or SE bars are given would not be accurate.
Potential Effectiveness as a Teaching Tool
Instructor should find it easy to write assessment questions for this activity.
Adequately demonstrates concept of overlapping confidence intervals without formulas.
This game promotes discovery learning.
This game does not have any direction about how to begin to think about the overlap/non-overlap of CI/SE bar and how that relates to a p-value from a two sample test. Some background information would help students to just not “guess” at the first two rounds and would enhance the conceptual knowledge they could potentially learn from the game.
The differences between the means for each of the different p-values is sometimes very small. The instructor needs to go through the material several times before the feel of the gradations of differences is somewhat obvious.
Ease of Use for Both Students and Faculty
Most students like a game orientated approach to learning. Appropriate, simple design. Accompanying guide is necessary and contains very good references for both instructors and students.
I suspect that many students will have difficulty understanding how to get started on this game and even authoring a “take-home” message from the game. Instructor input is imperative.