What is Probability?

We all encounter uncertainty every day. Will it rain this afternoon? What are the chances a flipped coin lands heads? If you draw a card from a shuffled deck, how likely is it to be an ace? These questions all share a common structure: they ask about the likelihood of some outcome occurring in a situation we cannot predict with certainty.

Probability is the mathematical framework we use to quantify that uncertainty. It assigns a number between 0 and 1 to every possible outcome (or collection of outcomes) of a random process, where 0 means the outcome is impossible and 1 means it is certain. Everything in between tells us how likely an outcome is relative to all others.

Before we can assign probabilities, however, we need precise language for describing what we are measuring. The first part of this chapter is devoted entirely to building that language. It may seem like a lot of terminology upfront, but every definition here will be used constantly throughout the rest of these notes, so it is worth getting comfortable with these ideas early.

Experiments and Outcomes

The starting point for any probability problem is what we call a random experiment — a process whose outcome we cannot know in advance, but whose set of all possible outcomes we can describe completely.

Common examples of random experiments include: flipping a coin, rolling a die, drawing a card from a shuffled deck, measuring the time until a light bulb fails, or recording the number of customers who arrive at a store in an hour. In each case we know what could happen, even if we do not know what will happen.

Each distinct result of an experiment is called an outcome or sample point. The collection of all possible outcomes is called the sample space.

Describing the sample space correctly is one of the most important steps in solving any probability problem, and it is a step that students frequently rush. A poorly defined sample space leads to errors that are very difficult to track down later, so take care here.

Events

When we ask a probability question, we are rarely interested in one specific outcome. More often we ask about a set of outcomes—"what is the probability of rolling an even number?" or "what is the probability that at least one flip is heads?" Such a collection of outcomes is called an event.

Two special events deserve names of their own:

• The certain event is the entire sample space S itself. It always occurs, because whatever happens is by definition an element of S.
• The impossible event is the empty set ∅. It never occurs, because no outcome is an element of the empty set.

Assigning Probability

We now have the vocabulary to say what we are measuring. The next step is to actually assign numbers—probabilities—to events. There are several ways to build up to this, but the most general and rigorous starting point is the set of rules (called axioms) that any assignment of probabilities must satisfy.

These axioms may look abstract. Think of them as ground rules: they do not tell you how to find the probability of an event—that depends on the problem—but they tell you what any valid answer must look like. Any formula or method you use to compute probabilities must produce answers consistent with these three rules. If it does not, something has gone wrong.

Classical Probability

For experiments where every outcome in the sample space is equally likely, there is a beautifully simple formula for computing probabilities.

This is the formula most people learn first when studying probability, and it works perfectly—as long as all outcomes truly are equally likely. The phrase "equally likely" is doing a lot of work here. A fair coin has equally likely outcomes; a bent coin does not. A well-shuffled deck has equally likely draws; a stacked deck does not. Always ask yourself whether the equally-likely assumption is justified before using this formula.

Basic Properties of Probability

From the three Kolmogorov axioms we can derive a number of useful properties. These are not new rules—they all follow logically from the axioms—but they come up so frequently that it is worth listing them explicitly.

What's Coming

This section introduced the essential vocabulary and the classical probability formula. Here is a brief preview of how the rest of Chapter 1 develops these ideas:

• 1.2 Sample Spaces & Events — We look more carefully at how to construct sample spaces for multi-step experiments, and how set operations (union, intersection, complement) translate into probability statements.
• 1.3 Basic Probability Rules — We prove and apply the complement rule, the addition rule, and the inclusion-exclusion principle, with plenty of worked examples.
• 1.4 Counting Techniques — For experiments with large sample spaces, listing every outcome is impractical. We develop systematic counting methods (permutations, combinations) that let us apply the classical formula without enumeration.
• 1.5 Conditional Probability — How does partial information change our probability estimates? Conditional probability formalizes the idea of updating beliefs when we learn that some event has already occurred.
• 1.6 Bayes' Theorem — One of the most celebrated results in all of mathematics. Bayes' theorem lets us "reverse" conditional probabilities and is the foundation of an entire approach to statistics.
• 1.7 Independence — What does it really mean for two events to have nothing to do with each other? The formal definition of independence is more subtle than it first appears.