Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Users Search
Help Dashboard Sign In Sign Up
Q&A Meta

Review Suggested Edit

You can't approve or reject suggested edits because you haven't yet earned the Edit Posts ability.
View post

Approved.
This suggested edit was approved and applied to the post over 3 years ago by DNB‭.

16 / 255 
How do I prove Simpson's Paradox, scilicet $P(A|B) > P(A|B^C)$?
  • $\forall a,b,c,d > 0$, $a < b, c< d \implies [0 \le ab < cd](https://math.stackexchange.com/q/3632752)$.
  • 1. The blue, orange and purple inequalities below are all in the _OPPOSITE _direction as $P(A|B) > P(A|B^C)$! I can multiply the blue and orange inequalities together, but their product is in the opposite direction too!
  • 2. I can't simply multiple the purple inequality by the green inequality, because the green inequality's in the _SAME_ direction as $P(A|B) > P(A|B^C)$.
  • I modified this example to make it easier to understand.
  • >### Example 2.8.3 (Simpson's paradox).
  • >Two doctors, Dr. Good and Dr. Bad, each
  • perform two types of surgeries: cardiac surgery and Band-Aid removal. Each surgery
  • can be either a success or a failure. The two doctors' respective records are given in
  • the following tables, and shown graphically in Figure 2.6, where white dots represent
  • successful surgeries and black dots represent failed surgeries.
  • ![Image alt text](https://math.codidact.com/uploads/dou1zM7NExzbkpuQjJfMxrFW)
  • >Dr. Good had a higher success rate than Dr. Bad in heart surgeries: 70 out of 90
  • versus 2 out of 10. Dr. Good also had a higher success rate in Band-Aid removal:
  • 10 out of 10 versus 81 out of 90. But if we aggregate across the two types of surgeries
  • to compare overall surgery success rates, Dr. Good was successful in 80 out of
  • 100 surgeries while Dr. Bad was successful in 83 out of 100 surgeries: Dr. Bad's
  • overall success rate is higher!
  • ![Image alt text](https://math.codidact.com/uploads/5dRcL5csf7kaWwy6sEhVhqzn)
  • >**FIGURE 2.6**
  • An example of Simpson's paradox. White dots represent successful surgeries and
  • black dots represent failed surgeries. Dr. Good is better in both types of surgery
  • but has a lower overall success rate, because he is performing the harder type of
  • surgery much more often than Dr. Bad is.
  • >
  • >What's happening is that Dr. Good, presumably due to his reputation as the
  • superior doctor, is performing a greater number of cardiac surgeries, which are inherently
  • riskier than Band-Aid removals. His overall success rate is lower not because
  • of lesser skill on any particular type of surgery, but because a larger fraction of his
  • surgeries are risky.
  • >
  • >Let's use event notation to make this precise. For events A, B, and C, we say that
  • we have a Simpson's paradox if
  • >
  • >$\begin{align}
  • P(A|B,C) & < P(A|B^C,C) \\
  • $P(A|B,C^cC & < P(A|B^C, C^C)$; \\
  • but $P(A|B) & > P(A|B^C)
  • \end{align}$.
  • >
  • >In this case, let A be the event of a successful surgery, B be the event that Dr. Bad
  • is the surgeon, and C be the event that the surgery is a cardiac surgery. The conditions
  • for Simpson's paradox are ful filled because the probability of a successful surgery
  • is lower under Dr. Bad than under Dr. Good whether we condition on heart
  • surgery or on Band-Aid removal, but the overall probability of success is higher for
  • Dr. Bad.
  • ![Image alt text](https://i.imgur.com/4tbwpq8.png)
  • Blitzstein. *Introduction to Probability* (2019 2 ed). pp 76-78.
  • $\forall a,b,c,d > 0, a<b, c<d \implies$ [$0 \le ab < cd$](https://math.stackexchange.com/q/3632752).
  • 1. The blue, orange and purple inequalities below are all in the _OPPOSITE_ direction as $P(A|B)>P(A|B^C)$! I can multiply the blue and orange inequalities together, but their product is in the opposite direction too!
  • 2. I can't simply multiple the purple inequality by the green inequality, because the green inequality's in the _SAME_ direction as $P(A|B)>P(A|B^C)$.
  • I modified this example to make it easier to understand.
  • >### Example 2.8.3 (Simpson's paradox).
  • >Two doctors, Dr. Good and Dr. Bad, each perform two types of surgeries: cardiac surgery and Band-Aid removal. Each surgery can be either a success or a failure. The two doctors' respective records are given in the following tables, and shown graphically in Figure 2.6, where white dots represent
  • successful surgeries and black dots represent failed surgeries.
  • ![Image alt text](https://math.codidact.com/uploads/dou1zM7NExzbkpuQjJfMxrFW)
  • >Dr. Good had a higher success rate than Dr. Bad in heart surgeries: 70 out of 90 versus 2 out of 10. Dr. Good also had a higher success rate in Band-Aid removal: 10 out of 10 versus 81 out of 90. But if we aggregate across the two types of surgeries to compare overall surgery success rates, Dr. Good was successful in 80 out of 100 surgeries while Dr. Bad was successful in 83 out of 100 surgeries: Dr. Bad's overall success rate is higher!
  • ![Image alt text](https://math.codidact.com/uploads/5dRcL5csf7kaWwy6sEhVhqzn)
  • >**FIGURE 2.6**
  • An example of Simpson's paradox. White dots represent successful surgeries and
  • black dots represent failed surgeries. Dr. Good is better in both types of surgery but has a lower overall success rate, because he is performing the harder type of surgery much more often than Dr. Bad is.
  • >
  • >What's happening is that Dr. Good, presumably due to his reputation as the
  • superior doctor, is performing a greater number of cardiac surgeries, which are inherently riskier than Band-Aid removals. His overall success rate is lower not because of lesser skill on any particular type of surgery, but because a larger fraction of his surgeries are risky.
  • >
  • >Let's use event notation to make this precise. For events A, B, and C, we say that we have a Simpson's paradox if
  • \begin{align*}
  • P(A|B,C) &<P(A|B^C,C)\newline
  • P(A|B,C^C) &<P(A|B^C, C^C);\newline
  • \text{but }P(A|B) &>P(A|B^C).
  • \end{align*}
  • >In this case, let A be the event of a successful surgery, B be the event that Dr. Bad is the surgeon, and C be the event that the surgery is a cardiac surgery. The conditions for Simpson's paradox are fulfilled because the probability of a successful surgery is lower under Dr. Bad than under Dr. Good whether we condition on heart surgery or on Band-Aid removal, but the overall probability of success is higher for Dr. Bad.
  • ![Image alt text](https://i.imgur.com/4tbwpq8.png)
  • Blitzstein. *Introduction to Probability* (2019 2 ed). pp 76-78.
probability
probability

Suggested over 3 years ago by JRN‭