11  Conditional Probability and Bayes’ Theorem

The previous chapter’s warning left off with a real problem: the simple multiplication rule \(P(A \text{ and } B) = P(A) \times P(B)\) only works for independent events, and most interesting business questions involve events that are not independent. Whether a customer is satisfied genuinely depends on whether they bought online or in-store. Whether a transaction is fraudulent depends on other signals in the data. Conditional probability is the tool for handling exactly this: the probability of one event, given that another has already happened. This chapter builds conditional probability up to Bayes’ Theorem, the formula for revising a probability once new evidence arrives, arguably the single most widely used idea from probability theory in modern data analytics.

11.1 What Is Conditional Probability?

The conditional probability of event \(B\) given that event \(A\) has occurred, written \(P(B \mid A)\), restricts attention to only the outcomes where \(A\) happened, and asks what fraction of those also satisfy \(B\).

\[P(B \mid A) = \frac{P(A \text{ and } B)}{P(A)} \qquad \text{(provided } P(A) > 0\text{)}\]

Example

Returning to the same 200-customer table from Chapter 9:

Satisfied Not Satisfied Total
Online 70 30 100
In-Store 60 40 100
Total 130 70 200

\[P(\text{Satisfied} \mid \text{Online}) = \frac{P(\text{Online and Satisfied})}{P(\text{Online})} = \frac{0.35}{0.50} = 0.70\]

\[P(\text{Satisfied} \mid \text{In-Store}) = \frac{60/200}{100/200} = \frac{60}{100} = 0.60\]

Online customers are satisfied 70% of the time, versus 60% for in-store customers, a real difference that the overall figure \(P(\text{Satisfied}) = 0.65\) completely hides. This is exactly why conditional probability matters: it reveals relationships that a single overall probability cannot.

11.2 The General Multiplication Rule

Rearranging the conditional probability formula gives the general multiplication rule, which works whether or not \(A\) and \(B\) are independent, unlike the simplified version from Chapter 9.

\[P(A \text{ and } B) = P(A) \times P(B \mid A)\]

Example

\[P(\text{Online and Satisfied}) = P(\text{Online}) \times P(\text{Satisfied} \mid \text{Online}) = 0.50 \times 0.70 = 0.35\]

This matches the table exactly (\(\frac{70}{200} = 0.35\)), and it is the version of the multiplication rule to reach for whenever independence hasn’t been confirmed.

11.3 Independence, Formally

Chapter 9 defined independence informally; conditional probability gives the precise definition. Events \(A\) and \(B\) are independent exactly when knowing \(A\) happened doesn’t change the probability of \(B\):

\[A \text{ and } B \text{ are independent} \iff P(B \mid A) = P(B)\]

If \(P(B \mid A) \neq P(B)\), the events are dependent, and the simple multiplication rule from Chapter 9 does not apply.

Example

\(P(\text{Satisfied} \mid \text{Online}) = 0.70\), but \(P(\text{Satisfied}) = 0.65\). Since these are not equal, purchase channel and satisfaction are confirmed dependent, exactly what the independence check in Chapter 9 already suggested.

11.4 Bayes’ Theorem

Bayes’ Theorem answers a question that comes up constantly in data analytics: we know \(P(B \mid A)\), but what we actually want is \(P(A \mid B)\), the reverse direction. It is derived by setting the two versions of the general multiplication rule equal to each other (\(P(A \text{ and } B) = P(A)P(B\mid A) = P(B)P(A \mid B)\)) and solving for \(P(A \mid B)\).

\[P(A \mid B) = \frac{P(B \mid A) \times P(A)}{P(B)}\]

\(P(A)\) is called the prior probability (what we believed before seeing the evidence \(B\)), and \(P(A \mid B)\) is the posterior probability (the updated belief after seeing it). When \(P(B)\) isn’t given directly, it is computed by summing over every way \(B\) can happen:

\[P(B) = P(B \mid A) \times P(A) + P(B \mid A') \times P(A')\]

Example

A factory’s items are defective 2% of the time (\(P(\text{Defective}) = 0.02\), the prior). An automated inspection machine correctly flags a truly defective item 95% of the time (\(P(\text{Flag} \mid \text{Defective}) = 0.95\)), but also incorrectly flags a good item 3% of the time (\(P(\text{Flag} \mid \text{Good}) = 0.03\)).

Question: If an item gets flagged, what’s the probability it is actually defective, \(P(\text{Defective} \mid \text{Flag})\)?

First find \(P(\text{Flag})\), the overall chance any item gets flagged, by summing both ways it can happen:

\[P(\text{Flag}) = (0.95)(0.02) + (0.03)(0.98) = 0.019 + 0.0294 = 0.0484\]

Then apply Bayes’ Theorem:

\[P(\text{Defective} \mid \text{Flag}) = \frac{(0.95)(0.02)}{0.0484} = \frac{0.019}{0.0484} \approx 0.393\]

Only about 39.3% of flagged items are actually defective, even though the inspection machine sounds highly accurate (95% detection rate). This is not a contradiction, it happens because true defects are rare (a 2% prior), so the far larger pool of good items, even with just a 3% false-positive rate, still produces more false alarms in absolute terms than the small number of true defects produces correct alarms.

This surprising result is often called the base rate fallacy when people get it wrong, mistaking a test’s accuracy (\(P(\text{Flag} \mid \text{Defective}) = 95\%\)) for the answer to the actual question asked (\(P(\text{Defective} \mid \text{Flag})\)). The same structure appears constantly: medical test results, spam filters, fraud detection, credit risk scoring. Whenever the event being detected is rare, always compute the full Bayes’ Theorem calculation rather than trusting the test’s stated accuracy at face value.

11.5 Where This Shows Up

  • Quality control: Interpreting inspection or sensor alerts correctly, as in the example above.
  • Spam and fraud detection: Updating the probability a message or transaction is malicious given specific flagged signals.
  • Medical and diagnostic testing: The original context Bayes’ Theorem is most famous for, updating disease probability given a test result.
  • Credit and risk scoring: Updating the probability of default given new information about a borrower.
  • A/B testing and Bayesian inference: An entire branch of inferential statistics, covered later in this book, builds directly on treating probability as an updatable belief exactly the way this chapter does.
Recap

Chapters 9 and 10 built probability from the ground up: sample spaces and events, the three ways probability is defined, the addition and multiplication rules, conditional probability, and Bayes’ Theorem for reversing a conditional probability once evidence arrives. This is the foundation Module III continues to build on: the next topics extend these ideas into full probability distributions (Binomial, Poisson, Normal), which describe entire ranges of outcomes rather than single events, and set up the inferential statistics that follow.

Summary

Concept Description
Core Idea
Conditional Probability The probability of one event given that another has already occurred
General Multiplication Rule P(A and B) equals P(A) times P(B given A); valid for dependent and independent events alike
Independence (Formal Definition) A and B are independent exactly when P(B given A) equals P(B)
Bayes' Theorem
Bayes' Theorem A formula for reversing a conditional probability: P(A given B) from P(B given A) and P(A)
Prior Probability The initial probability of an event before new evidence is taken into account
Posterior Probability The updated probability of an event after new evidence has been incorporated
Law of Total Probability P(B) computed by summing P(B given A) times P(A) over every way B can occur
Using It Correctly
Base Rate Fallacy Mistaking a test's accuracy for the probability the event is real, especially when the event is rare
Applications of Bayes' Theorem Quality control, spam and fraud detection, medical testing, credit risk, and Bayesian A/B testing