## 13.1 Background and Probability Review

We will start by defining some terms and reviewing basic ideas from
probability. We assume that the reader is already familiar with
basic probability concepts; readers needing a more complete introduction to this
topic should consult a textbook such as Sheldon Ross’s *Introduction to
Probability Models* (2002).

A *random variable* is a value chosen by some random process. We
will generally use capital letters to denote random variables, with
exceptions made for a few Greek symbols that represent special random variables.
Random variables are always drawn from some domain, which can be either
discrete (e.g., a fixed set of possibilities) or continuous (e.g., the real
numbers ).
Applying a function to a random variable results in a
new random variable .

For example, the result of a roll of a die is a discrete random variable sampled from the set of events . Each event has a probability , and the sum of probabilities is necessarily one. We can take a continuous, uniformly distributed random variable and map it to a discrete random variable, choosing if

For lighting applications, we might want to define the probability of sampling illumination from each light in the scene based on the power from each source relative to the total power from all sources:

Notice that these also sum to 1.

The *cumulative distribution function* (CDF) of a random
variable is the probability that a value from the variable’s distribution is less
than or equal to some value :

For the die example, , since two of the six possibilities are less than or equal to 2.

### 13.1.1 Continuous Random Variables

In rendering, discrete random variables are less common than continuous random variables, which take on values over ranges of continuous domains (e.g., the real numbers, directions on the unit sphere, or the surfaces of shapes in the scene).

A particularly important random variable is the *canonical uniform
random variable*, which we will write as . This variable takes on all
values in its domain with equal probability. This particular
variable is important for two reasons. First, it is easy to generate a
variable with this distribution in software—most run-time libraries have a
pseudo-random number generator that does just that.
Second,
as we will show later, it is possible to generate samples from arbitrary
distributions by first starting with canonical uniform random variables and applying
an appropriate transformation. The technique described previously for
mapping from
to the six faces of a die gives a flavor of this technique in the discrete
case.

Another example of a continuous random variable is one that ranges over the
real numbers between 0 and 2, where the probability of its taking on any
particular value is proportional to the value :
it is twice as
likely for this random variable to take on a value around 0 as it is to
take one around 1, and so forth. The *probability density function*
(PDF) formalizes this idea: it describes the relative probability of a
random variable taking on a particular value. The PDF is the
derivative of the random variable’s CDF,

For uniform random variables, is a constant; this is a direct consequence of uniformity. For we have

PDFs are necessarily nonnegative and always integrate to 1 over their domains. Given an arbitrary interval in the domain, integrating the PDF gives the probability that a random variable lies inside the interval:

This follows directly from the first fundamental theorem of calculus and the definition of the PDF.

### 13.1.2 Expected Values and Variance

The *expected value* of a function is defined as the
average value of the function over some distribution of values over its
domain. In the next section, we will see how Monte Carlo integration
computes the expected values of arbitrary integrals. The expected value over a
domain, , is defined as

As an example, consider the problem of finding the expected value of the cosine function between and , where is uniform. Because the PDF must integrate to 1 over the domain, , so

which is precisely the expected result. (Consider the graph of over to see why this is so.)

The *variance* of a function is the expected squared deviation of the function
from its expected value. Variance is a fundamental concept for quantifying
the error in a value estimated by a Monte Carlo algorithm. It provides a
precise way to quantify this error and measure how improvements to Monte
Carlo algorithms reduce the error in the final result. The variance of
a function is defined as

The expected value and variance have a few important properties that follow immediately from their respective definitions:

These properties, and some simple algebraic manipulation, yield an alternative expanded expression for the variance:

Thus, the variance is the expected value of the square minus the square of
the expected value. Given random variables that are *independent*,
variance also has the property that the sum of the variances is equal to
the variance of their sum: