Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs

Dashboard
Notifications
Mark all as read
Q&A

How do I prove Simpson's Paradox, scilicet $P(A|B) > P(A|B^C)$?

+0
−3

$\forall a,b,c,d > 0, a<b, c<d \implies$ $0 \le ab < cd$.

  1. The blue, orange and purple inequalities below are all in the OPPOSITE direction as $P(A|B)>P(A|B^C)$! I can multiply the blue and orange inequalities together, but their product is in the opposite direction too!

  2. I can't simply multiply the purple inequality by the green inequality, because the green inequality's in the SAME direction as $P(A|B)>P(A|B^C)$.

I modified this example to make it easier to understand.

Example 2.8.3 (Simpson's paradox).

Two doctors, Dr. Good and Dr. Bad, each perform two types of surgeries: cardiac surgery and Band-Aid removal. Each surgery can be either a success or a failure. The two doctors' respective records are given in the following tables, and shown graphically in Figure 2.6, where white dots represent successful surgeries and black dots represent failed surgeries.

Image alt text

Dr. Good had a higher success rate than Dr. Bad in heart surgeries: 70 out of 90 versus 2 out of 10. Dr. Good also had a higher success rate in Band-Aid removal: 10 out of 10 versus 81 out of 90. But if we aggregate across the two types of surgeries to compare overall surgery success rates, Dr. Good was successful in 80 out of 100 surgeries while Dr. Bad was successful in 83 out of 100 surgeries: Dr. Bad's overall success rate is higher!

Image alt text

FIGURE 2.6
An example of Simpson's paradox. White dots represent successful surgeries and black dots represent failed surgeries. Dr. Good is better in both types of surgery but has a lower overall success rate, because he is performing the harder type of surgery much more often than Dr. Bad is.

What's happening is that Dr. Good, presumably due to his reputation as the superior doctor, is performing a greater number of cardiac surgeries, which are inherently riskier than Band-Aid removals. His overall success rate is lower not because of lesser skill on any particular type of surgery, but because a larger fraction of his surgeries are risky.

Let's use event notation to make this precise. For events A, B, and C, we say that we have a Simpson's paradox if \begin{align*} P(A|B,C) &<P(A|B^C,C)\newline P(A|B,C^C) &<P(A|B^C, C^C);\newline \text{but }P(A|B) &>P(A|B^C). \end{align*} In this case, let A be the event of a successful surgery, B be the event that Dr. Bad is the surgeon, and C be the event that the surgery is a cardiac surgery. The conditions for Simpson's paradox are fulfilled because the probability of a successful surgery is lower under Dr. Bad than under Dr. Good whether we condition on heart surgery or on Band-Aid removal, but the overall probability of success is higher for Dr. Bad.

Image alt text

Blitzstein. Introduction to Probability (2019 2 ed). pp 76-78.

Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

0 answers

Sign up to answer this question »

This community is part of the Codidact network. We have other communities too — take a look!

You can also join us in chat!

Want to advertise this community? Use our templates!

Like what we're doing? Support us! Donate