10 Basic Probability Concepts
Modules I and II were entirely about data you already have in hand: organizing it, summarizing its center and spread, describing its shape. Probability is different, it is the mathematics of uncertainty, of what is likely to happen before it happens. It is also the bridge to everything ahead in this book: sampling, estimation, and hypothesis testing all rest on the ideas in this chapter and the next. This chapter covers sample spaces and events, the three ways probability is defined, and the addition and multiplication rules for combining probabilities.
10.1 What Is Probability?
The sample space is the set of every possible outcome of a random process, usually written \(S\). An event is any subset of the sample space, one outcome or a combination of outcomes that we are interested in. A probability is a number between 0 and 1 (or 0% and 100%) that measures how likely an event is: 0 means the event cannot happen, 1 means it is certain.
Probability is assigned in three different ways depending on the situation:
- Classical (a priori) probability: Used when all outcomes are equally likely and can be counted in advance, such as a fair coin or die. \(P(A) = \dfrac{\text{Number of outcomes favorable to } A}{\text{Total number of possible outcomes}}\).
- Empirical (relative frequency) probability: Used when outcomes are not equally likely or can’t be reasoned out in advance, so probability is estimated from observed data instead. \(P(A) = \dfrac{\text{Number of times } A \text{ occurred}}{\text{Total number of trials observed}}\). Most business probabilities, such as the probability a customer churns or a shipment arrives late, are empirical.
- Subjective probability: A degree of belief based on judgment and experience rather than counting or data, used when neither of the above is available, such as an expert’s estimate that a new product launch will succeed.
Example
A company surveys 200 customers on how they purchased (Online or In-Store) and whether they were satisfied with the purchase (Satisfied or Not Satisfied):
| Satisfied | Not Satisfied | Total | |
|---|---|---|---|
| Online | 70 | 30 | 100 |
| In-Store | 60 | 40 | 100 |
| Total | 130 | 70 | 200 |
This is empirical probability: nothing is assumed about how likely a customer is to be satisfied, it is read directly from what was observed. From this table, \(P(\text{Online}) = \frac{100}{200} = 0.50\) and \(P(\text{Satisfied}) = \frac{130}{200} = 0.65\). This same table is used throughout this chapter and the next.
10.2 The Complement Rule
The complement of an event \(A\), written \(A'\) (or \(\bar{A}\)), is “\(A\) does not happen.” Since an event either happens or it doesn’t, their probabilities always add to 1.
\[P(A') = 1 - P(A)\]
Example
Since \(P(\text{Satisfied}) = 0.65\), the probability a customer was not satisfied is \(P(\text{Not Satisfied}) = 1 - 0.65 = 0.35\), which matches the table directly (\(\frac{70}{200} = 0.35\)).
10.3 The Addition Rule
The addition rule finds the probability that at least one of two events happens, \(P(A \text{ or } B)\). It takes two forms depending on whether the events can happen at the same time.
Mutually exclusive events (cannot both happen at once, such as a customer being on exactly one subscription plan):
\[P(A \text{ or } B) = P(A) + P(B)\]
Events that can overlap (the general case): simply adding \(P(A)\) and \(P(B)\) would double-count the outcomes where both happen, so that overlap must be subtracted back out once.
\[P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)\]
Example
Mutually exclusive: A telecom company’s 500 customers are each on exactly one plan: 200 Basic, 180 Standard, 120 Premium. Since a customer cannot be on two plans at once, \(P(\text{Basic or Standard}) = \frac{200}{500} + \frac{180}{500} = 0.40 + 0.36 = 0.76\).
Overlapping: Using the Online/Satisfied table, “Online” and “Satisfied” can happen together (70 customers are both), so:
\[P(\text{Online or Satisfied}) = P(\text{Online}) + P(\text{Satisfied}) - P(\text{Online and Satisfied}) = 0.50 + 0.65 - 0.35 = 0.80\]
This checks out directly against the table: 160 of the 200 customers were either online, satisfied, or both (\(\frac{160}{200} = 0.80\)).
10.4 The Multiplication Rule
The multiplication rule finds the probability that both of two events happen, \(P(A \text{ and } B)\). For independent events, where one event’s outcome has no effect on the other’s probability, it is simply the product of the two probabilities.
\[P(A \text{ and } B) = P(A) \times P(B) \quad \text{(independent events only)}\]
When events are dependent, this simple product no longer works, that case needs conditional probability, covered in the next chapter.
Example
A machine is assembled from two components sourced from independent suppliers. Component A works with probability 0.95, and Component B works with probability 0.90, independently of A.
\[P(\text{Both work}) = 0.95 \times 0.90 = 0.855\]
There is an 85.5% chance the assembled machine works, even though each individual component is quite reliable on its own, a useful reminder that reliability compounds downward across independent parts.
The independent-events multiplication rule is one of the most commonly misapplied formulas in business analytics. Channel and satisfaction in our running example are not independent: \(P(\text{Online}) \times P(\text{Satisfied}) = 0.50 \times 0.65 = 0.325\), but the actual \(P(\text{Online and Satisfied}) = 0.35\). Applying the independent-events formula when events are actually dependent will silently give the wrong answer. Always check for independence, or use conditional probability instead, before multiplying.
Looking Ahead
The multiplication rule above only works when events are independent, and the warning in this chapter showed our own running example is not. The next chapter introduces conditional probability, the tool for handling dependent events correctly, and builds up to Bayes’ Theorem, one of the single most useful ideas in all of data analytics for updating a belief once new evidence arrives.
Summary
| Concept | Description |
|---|---|
| Core Idea | |
| Probability | A number between 0 and 1 measuring how likely an event is to occur |
| Sample Space | The set of every possible outcome of a random process |
| Event | Any subset of outcomes from the sample space that we are interested in |
| Defining Probability | |
| Classical Probability | Favorable outcomes over total outcomes, used when all outcomes are equally likely |
| Empirical Probability | Observed frequency of an event over total trials, used when probability must be estimated from data |
| Subjective Probability | A judgment-based probability used when neither counting nor data is available |
| Combining Events | |
| Complement Rule | The probability an event does not happen equals 1 minus the probability it does |
| Addition Rule (Mutually Exclusive) | For events that cannot both occur, P(A or B) is simply P(A) plus P(B) |
| Addition Rule (General/Overlapping) | For events that can overlap, P(A or B) is P(A) plus P(B) minus P(A and B), to avoid double counting |
| Multiplication Rule (Independent Events) | For independent events, P(A and B) is simply P(A) times P(B) |
| Dependence | |
| Checking Independence | Independence means P(A and B) equals P(A) times P(B); if not, the events are dependent |