Probability

An experiment is called a random experiment if it satisfies two conditions: it has more than one possible outcome, and it is not possible to predict the outcome in advance, even though all the possible outcomes are known beforehand. Tossing a coin, throwing a die, and drawing a card from a well-shuffled deck are all random experiments.

The set of all possible outcomes of a random experiment is called its sample space, denoted by S. Each element of S is called a sample point or an outcome. The number of elements in S is written as n(S).

• Tossing a coin once: S = {H, T}, so n(S) = 2.
• Throwing a die once: S = {1, 2, 3, 4, 5, 6}, so n(S) = 6.
• Tossing a coin twice (or two coins once): S = {HH, HT, TH, TT}, so n(S) = 4.

An event is simply a subset of the sample space S. An event is said to occur if the outcome of the experiment is one of the sample points belonging to that subset.

Worked Example: A coin is tossed three times. Write the sample space, and find the number of outcomes in the event A = "exactly two heads appear".

Each toss has 2 possible outcomes (H or T), and there are 3 tosses, so n(S) = 2³ = 8. Listing them:

S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

The outcomes with exactly two heads are HHT, HTH and THH. So A = {HHT, HTH, THH} and n(A) = 3.

Depending on how many sample points an event contains, events are classified as follows.

• The impossible event is the empty set φ. It contains no sample point and can never occur, so it corresponds to probability 0.
• The sure (or certain) event is the whole sample space S itself. It always occurs, since it contains every possible outcome, so it corresponds to probability 1.
• A simple (or elementary) event is an event that has exactly one sample point, that is, a singleton subset of S, such as {3} in a die throw.
• A compound event is an event that has more than one sample point; it can be expressed as a union of two or more elementary events, such as "getting an even number" = {2, 4, 6} on a die.

Every event A has a corresponding complementary event, written A′ (or "not A"), consisting of all outcomes of S that do NOT belong to A. Thus A′ = S − A, and A and A′ together make up the whole sample space, with no outcome common to both.

Worked Example: A die is thrown once. Let A be the event "a number greater than 4 appears". List A and its complement A′, and state whether A is simple or compound.

S = {1, 2, 3, 4, 5, 6}. A = {5, 6}, since 5 and 6 are the only numbers greater than 4. As A has two sample points, it is a compound event. Its complement is A′ = S − A = {1, 2, 3, 4}.

Since events are sets, the usual set operations describe how events combine.

• A ∪ B ("A union B") is the event that A occurs OR B occurs (or both) — it consists of all outcomes that belong to A, to B, or to both.
• A ∩ B ("A intersection B") is the event that A occurs AND B occurs — it consists of all outcomes common to both A and B.
• A − B (equivalently A ∩ B′) is the event that A occurs but B does not.

Two events A and B are called mutually exclusive (or disjoint) if they cannot occur simultaneously, that is, A ∩ B = φ. For example, on a single die throw, "getting an even number" and "getting an odd number" are mutually exclusive because no outcome is both even and odd.

Events E₁, E₂, ..., E_n are called exhaustive events if their union is the entire sample space, that is, E₁ ∪ E₂ ∪ ... ∪ E_n = S. This means at least one of them must occur every time the experiment is performed.

When a set of events is BOTH pairwise mutually exclusive AND exhaustive, they are called mutually exclusive and exhaustive events — exactly one of them occurs on every trial. The elementary events of any sample space always form such a set.

Two useful identities (De Morgan's laws) are: (A ∪ B)′ = A′ ∩ B′, and (A ∩ B)′ = A′ ∪ B′.

Worked Example: A die is thrown once. Let A = {1, 2, 3}, B = {3, 4, 5}, C = {5, 6}. Examine each pair for mutual exclusiveness, and check whether A, B, C are exhaustive.

A ∩ B = {3}, which is not empty, so A and B are NOT mutually exclusive. B ∩ C = {5}, not empty, so B and C are NOT mutually exclusive. A ∩ C = φ (A has no element in common with C), so A and C ARE mutually exclusive. Finally, A ∪ B ∪ C = {1, 2, 3, 4, 5, 6} = S, so A, B, C together are exhaustive events.

In the axiomatic approach, probability is defined as a function P that assigns to every event A of a sample space S a real number P(A), called the probability of A, satisfying the following axioms:

1. 0 ≤ P(A) ≤ 1 for every event A.
2. P(S) = 1.
3. If A and B are mutually exclusive events, then P(A ∪ B) = P(A) + P(B). More generally, if E₁, E₂, ..., E_n are pairwise mutually exclusive events, then P(E₁ ∪ E₂ ∪ ... ∪ E_n) = P(E₁) + P(E₂) + ... + P(E_n).

In practice, this means we first assign a probability p_i to every elementary event {ω_i} of the sample space S = {ω₁, ω₂, ..., ω_n}, such that 0 ≤ p_i ≤ 1 for each i, and the sum of all the p_i equals 1. The probability of any compound event A is then found by adding the probabilities of the elementary events that make it up. If all outcomes are equally likely, each p_i = 1/n(S).

Two important results follow directly from the axioms: since φ and S are mutually exclusive with φ ∪ S = S, we get P(φ) = 0; and since A and A′ are mutually exclusive with A ∪ A′ = S, we get P(A) + P(A′) = 1, that is, P(A′) = 1 − P(A).

Worked Example: A sample space is S = {ω₁, ω₂, ω₃}. It is given that P(ω₁) = P(ω₂) and P(ω₃) = 2P(ω₁). Find the probability of each elementary event.

Let P(ω₁) = p. Then P(ω₂) = p and P(ω₃) = 2p. By the axioms, the probabilities of all elementary events must add up to 1:

p + p + 2p = 1, so 4p = 1, giving p = 1/4.

Hence P(ω₁) = 1/4, P(ω₂) = 1/4 and P(ω₃) = 1/2.

When a random experiment has outcomes that are all equally likely to occur — such as a fair coin, an unbiased die, or a well-shuffled deck of cards — the axiomatic approach reduces to the familiar classical definition of probability:

P(A) = n(A) / n(S), where n(A) is the number of outcomes favourable to event A, and n(S) is the total number of outcomes in the sample space.

This formula is only valid when the outcomes are equally likely; it must not be used otherwise. It automatically satisfies the axioms: since 0 ≤ n(A) ≤ n(S), we get 0 ≤ P(A) ≤ 1, and P(S) = n(S)/n(S) = 1.

A very useful trick for "at least one" type problems is to work through the complement: P(at least one) = 1 − P(none).

Worked Example: Two fair dice are thrown together. Find the probability that the sum of the numbers appearing on them is 8.

Each die has 6 faces, so the sample space of ordered pairs (first die, second die) has n(S) = 6 × 6 = 36 equally likely outcomes.

The pairs whose sum is 8 are: (2,6), (3,5), (4,4), (5,3), (6,2). That gives n(A) = 5.

Therefore P(A) = 5/36.

Worked Example (complement trick): A coin is tossed twice. Find the probability of getting at least one head.

S = {HH, HT, TH, TT}, so n(S) = 4. The only outcome with NO head is TT, so P(no head) = 1/4.

By the complement rule, P(at least one head) = 1 − 1/4 = 3/4.

For any two events A and B of a sample space S, the addition theorem (or addition rule) of probability states:

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

This subtracts P(A ∩ B) because outcomes common to both A and B would otherwise be counted twice — once in P(A) and once again in P(B).

If A and B are mutually exclusive (A ∩ B = φ), then P(A ∩ B) = 0, and the rule simplifies to the special case used in Axiom 3:

P(A ∪ B) = P(A) + P(B) (when A, B are mutually exclusive)

If, further, A and B are exhaustive as well (A ∪ B = S), then P(A ∪ B) = 1, so P(A) + P(B) = 1.

For three events A, B, C, the rule extends to:

P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) − P(B ∩ C) − P(A ∩ C) + P(A ∩ B ∩ C)

Two useful related results, both following from the addition theorem and De Morgan's law, are:

• P(A occurs but not B) = P(A ∩ B′) = P(A) − P(A ∩ B).
• P(neither A nor B occurs) = P(A′ ∩ B′) = 1 − P(A ∪ B).

Worked Example: A card is drawn from a well-shuffled deck of 52 playing cards. Find the probability that it is a king or a heart.

Let A = "the card is a king" and B = "the card is a heart". There are 4 kings, so P(A) = 4/52. There are 13 hearts, so P(B) = 13/52. The king of hearts is the single card common to both events, so P(A ∩ B) = 1/52.

By the addition theorem: P(A ∪ B) = 4/52 + 13/52 − 1/52 = 16/52 = 4/13.

Worked Example (mutually exclusive case): A fair die is thrown once. Let A = "an even number appears" = {2, 4, 6} and B = "the number 5 appears" = {5}. Since A ∩ B = φ, A and B are mutually exclusive, so:

P(A ∪ B) = P(A) + P(B) = 3/6 + 1/6 = 4/6 = 2/3.