13.1 Background and Probability Review
We will start by defining some terms and reviewing basic ideas from probability. We assume that the reader is already familiar with basic probability concepts; readers needing a more complete introduction to this topic should consult a textbook such as Sheldon Ross’s Introduction to Probability Models (2002).
A random variable is a value chosen by some random process. We will generally use capital letters to denote random variables, with exceptions made for a few Greek symbols that represent special random variables. Random variables are always drawn from some domain, which can be either discrete (e.g., a fixed, finite set of possibilities) or continuous (e.g., the real numbers ). Applying a function to a random variable results in a new random variable .
For example, the result of a roll of a die is a discrete random variable sampled from the set of events . Each event has a probability , and the sum of probabilities is necessarily one. We can take a continuous, uniformly distributed random variable and map it to a discrete random variable, choosing if
For lighting applications, we might want to define the probability of sampling illumination from each light in the scene based on the power from each source relative to the total power from all sources:
Notice that these also sum to 1.
The cumulative distribution function (CDF) of a random variable is the probability that a value from the variable’s distribution is less than or equal to some value :
For the die example, , since two of the six possibilities are less than or equal to 2.
13.1.1 Continuous Random Variables
In rendering, discrete random variables are less common than continuous random variables, which take on values over ranges of continuous domains (e.g., the real numbers, directions on the unit sphere, or the surfaces of shapes in the scene).
A particularly important random variable is the canonical uniform random variable, which we will write as . This variable takes on all values in its domain with equal probability. This particular variable is important for two reasons. First, it is easy to generate a variable with this distribution in software—most run-time libraries have a pseudo-random number generator that does just that. Second, as we will show later, it is possible to generate samples from arbitrary distributions by first starting with canonical uniform random variables and applying an appropriate transformation. The technique described previously for mapping from to the six faces of a die gives a flavor of this technique in the discrete case.
Another example of a continuous random variable is one that ranges over the real numbers between 0 and 2, where the probability of its taking on any particular value is proportional to the value : it is twice as likely for this random variable to take on a value around 0 as it is to take one around 1, and so forth. The probability density function (PDF) formalizes this idea: it describes the relative probability of a random variable taking on a particular value. The PDF is the derivative of the random variable’s CDF,
For uniform random variables, is a constant; this is a direct consequence of uniformity. For we have
PDFs are necessarily nonnegative and always integrate to 1 over their domains. Given an arbitrary interval in the domain, integrating the PDF gives the probability that a random variable lies inside the interval:
This follows directly from the first fundamental theorem of calculus and the definition of the PDF.
13.1.2 Expected Values and Variance
The expected value of a function is defined as the average value of the function over some distribution of values over its domain. In the next section, we will see how Monte Carlo integration computes the expected values of arbitrary integrals. The expected value over a domain, , is defined as
As an example, consider the problem of finding the expected value of the cosine function between and , where is uniform. Because the PDF must integrate to 1 over the domain, , so
which is precisely the expected result. (Consider the graph of over to see why this is so.)
The variance of a function is the expected squared deviation of the function from its expected value. Variance is a fundamental concept for quantifying the error in a value estimated by a Monte Carlo algorithm. It provides a precise way to quantify this error and measure how improvements to Monte Carlo algorithms reduce the error in the final result. The variance of a function is defined as
The expected value and variance have a few important properties that follow immediately from their respective definitions:
These properties, and some simple algebraic manipulation, yield an alternative expanded expression for the variance:
Thus, the variance is the expected value of the square minus the square of the expected value. Given random variables that are independent, variance also has the property that the sum of the variances is equal to the variance of their sum: