## 6.4 Realistic Cameras

The thin lens model makes it possible to render images with blur due to
depth of field, but it is a fairly rough approximation of actual camera
lens systems, which are comprised of a series of multiple *lens
elements*, each of which modifies the distribution of radiance passing
through it.
(Figure 6.15 shows a cross section of a
22-mm focal length wide-angle lens with eight elements.)
Even basic cell phone cameras tend to have on the order of five individual
lens elements, while DSLR lenses may have ten or more. In general, more
complex lens systems with larger numbers of lens elements can create
higher quality images than simpler lens systems.

`scenes/lenses/wide.22mm.dat`in the

`pbrt`distribution). The lens coordinate system has the film plane perpendicular to the axis and located at . The lenses are to the left, along negative , and then the scene is to the left of the lenses. The aperture stop, indicated by the thick black lines in the middle of the lens system, blocks rays that hit it. In many lens systems, the size of the aperture stop can be adjusted to trade off between shorter exposure times (with larger apertures) and more depth of field (with smaller apertures).

This section discusses the implementation of `RealisticCamera`, which
simulates the focusing of light through lens systems like the one in
Figure 6.15 to render images like
Figure 6.16. Its implementation is based on ray
tracing, where the camera follows ray paths through the lens elements,
accounting for refraction at the interfaces between media (air, different types
of glass) with different indices of refraction, until the ray path either exits the
optical system or until it is absorbed by the aperture stop or lens housing. Rays leaving the front
lens element represent samples of the camera’s response profile and can be used
with integrators that estimate the incident radiance along arbitrary rays, such as the `SamplerIntegrator`.
The `RealisticCamera` implementation is in the files
`cameras/realistic.h` and
`cameras/realistic.cpp`.

In addition to the usual transformation to place the camera in the scene,
the `Film`, and the shutter open and close times, the
`RealisticCamera` constructor takes a filename for a lens system
description file, the distance to the desired plane of focus, and a
diameter for the aperture stop. The effect of the `simpleWeighting`
parameter is described later, in Section 13.6.6, after
preliminaries related to Monte Carlo integration in Chapter 13 and the
radiometry of image formation in
Section 6.4.7.

After loading the lens description file from disk, the constructor adjusts
the spacing between the lenses and the film so that the plane of focus is
at the desired depth, `focusDistance`, and then precomputes some
information about which areas of the lens element closest to the film carry
light from the scene to the film, as seen from various points on the film plane.
After background material has been introduced, the fragments
<<Compute lens–film distance for given focus distance>> and
<<Compute exit pupil bounds at sampled points on the film>> will be
defined in Sections 6.4.4 and 6.4.5,
respectively.

### 6.4.1 Lens System Representation

A lens system is made from a series of lens elements, where each element is generally some form of glass. A lens system designer’s challenge is to design a series of elements that form high-quality images on a film or sensor subject to limitations of space (e.g., the thickness of mobile phone cameras is very limited in order to keep phones thin), cost, and ease of manufacture.

It’s easiest to manufacture lenses with cross sections that are spherical,
and lens systems are generally symmetric around the *optical axis*,
which is conventionally denoted by . We will assume both of these
properties in the remainder of this section.
Lens systems are defined using a coordinate
system where the film is aligned with the plane and lenses are to the
left of the film, along the axis.

Lens systems are commonly represented in terms of the series of interfaces between the individual lens elements (or air) rather than having an explicit representation of each element. Table 6.1 shows the quantities that define each interface. The last entry in the table defines the rightmost interface, which is shown in Figure 6.17: it’s a section of a sphere with radius equal to the curvature radius. The thickness of an element is the distance along to the next element to the right (or to the film plane), and the index of refraction is for the medium to the right of the interface. The element’s extent above and below the axis is set by the aperture diameter.

Curvature Radius | Thickness | Index of Refraction | Aperture Diameter |
---|---|---|---|

35.98738 | 1.21638 | 1.54 | 23.716 |

11.69718 | 9.9957 | 1 | 17.996 |

13.08714 | 5.12622 | 1.772 | 12.364 |

22.63294 | 1.76924 | 1.617 | 9.812 |

71.05802 | 0.8184 | 1 | 9.152 |

0 | 2.27766 | 0 | 8.756 |

9.58584 | 2.43254 | 1.617 | 8.184 |

11.28864 | 0.11506 | 1 | 9.152 |

166.7765 | 3.09606 | 1.713 | 10.648 |

7.5911 | 1.32682 | 1.805 | 11.44 |

16.7662 | 3.98068 | 1 | 12.276 |

7.70286 | 1.21638 | 1.617 | 13.42 |

11.97328 | (depends on focus) | 1 | 17.996 |

The `LensElementInterface` structure represents a single lens element
interface.

The fragment <<Load element data from lens description file>>, not
included here, reads the lens elements and initializes the
`RealisticCamera::elementInterfaces` array. See comments in the source
code for details of the file format, which parallels the structure of
Table 6.1, and see the directory
`scenes/lenses` in the `pbrt` distribution for a number of example
lens descriptions.

Two adjustments are made to the values read from the file: first, lens
systems are traditionally described in units of millimeters, but `pbrt` assumes a scene measured in meters. Therefore, the fields other than the
index of refraction are scaled by . Second, the element’s diameter
is divided by two; the radius is a more convenient quantity to have at hand
in the code to follow.

Once the element interface descriptions have been loaded, it’s useful
to have a few values related to the lens system easily at hand.
`LensRearZ()` and `LensFrontZ()` return the depths of the
rear and front elements of the lens system, respectively.
Note that the returned depths are in camera space, not lens space, and
thus have positive values.

Finding the front element’s position requires summing all of the element
thicknesses (see Figure 6.18). This value isn’t
needed in any code that is in a performance-sensitive part of the system,
so recomputing it when needed is fine. If performance of this method was a
concern, it would be better to cache this value in the
`RealisticCamera`.

`RearElementRadius()` returns the aperture radius of the rear element
in meters.

### 6.4.2 Tracing Rays through Lenses

Given a ray starting from the film side of the lens system,
`TraceLensesFromFilm()` computes intersections with each element in
turn, terminating the ray and returning `false` if its path is blocked along
the way through the lens system. Otherwise it returns
`true` and initializes `*rOut` with the exiting ray in camera
space. During traversal, `elementZ` tracks the intercept of the
current lens element. Because the ray is starting from the film, the
lenses are traversed in reverse order compared to how they are stored in
`elementInterfaces`.

`rCamera`from camera to lens system space>> for (int i = elementInterfaces.size() - 1; i >= 0; --i) { const LensElementInterface &element = elementInterfaces[i]; <<Update ray from film accounting for interaction with

`element`>>

`rLens`from lens system space back to camera space>> return true; }

Because the camera points down the axis in `pbrt`’s camera space but
lenses are along , the components of the origin and direction of the
ray need to be negated. While this is a simple enough transformation that
it could be applied directly, we prefer an explicit `Transform` to make
the intent clear.

`rCamera`from camera to lens system space>>=

Recall from Figure 6.18 how the intercept of
elements is computed: because we are visiting the elements from
back-to-front, the element’s thickness must be subtracted from
`elementZ` to compute its intercept before the element interaction
is accounted for.

`element`>>=

Given the element’s axis intercept, the next step is to compute the
parametric value along the ray where it intersects the element
interface (or the plane of the aperture stop). For the aperture stop, a
ray–plane test (following Section 3.1.2) is used.
For spherical interfaces, `IntersectSphericalElement()` performs this
test and also returns the surface normal if an intersection is found; the
normal will be needed for computing the refracted ray direction.

The `IntersectSphericalElement()`
method is generally similar to `Sphere::Intersect()`, though it’s
specialized for the fact that the element’s center is along the axis
(and thus, the center’s and components are zero). The fragments
<<Compute `t0` and `t1` for ray–element intersection>>
and <<Compute surface normal of element at ray intersection point>>
aren’t included in the text here due to their similarity with the
`Sphere::Intersect()` implementation.

`t0`and

`t1`for ray–element intersection>>

There is, however, a subtlety in choosing which intersection point to return: the closest intersection with isn’t necessarily on the element interface; see Figure 6.19. For example, for a ray approaching from the scene and intersecting a concave lens (with negative curvature radius), the farther of the two intersections should be returned regardless of whether the closer one has . Fortunately, simple logic based on the ray direction and the curvature radius indicates which value to use.

Each lens element extends for some radius around the optical axis; if the intersection point with the element is outside this radius, then the ray will actually intersect the lens housing and terminate. In a similar fashion, if a ray intersects the aperture stop, it also terminates. Therefore, here we test the intersection point against the appropriate limit for the current element, either terminating the ray or updating its origin to the current intersection point if it survives.

If the current element is the aperture, the ray’s path isn’t affected by traveling through the element’s interface. For glass (or, forbid, plastic) lens elements, the ray’s direction changes at the interface as it goes from a medium with one index of refraction to one with another. (The ray may be passing from air to glass, from glass to air, or from glass with one index of refraction to a different type of glass with a different index of refraction.)

Section 8.2 discusses how a change in index
of refraction at the boundary between two media changes the direction of a
ray and the amount of radiance carried by the ray. (In this case, we can
ignore the change of radiance, as it cancels out if the ray is in the same
medium going into the lens system as it is when it exits—here, both are
air.) The `Refract()` function is defined in
Section 8.2.3; note that it expects that the incident
direction will point away from the surface, so the ray direction is negated
before being passed to it. This function returns `false` in the
presence of total internal reflection, in which case the ray path
terminates. Otherwise, the refracted direction is returned in `w`.

In general, some light passing through an interface like this is transmitted and some is reflected. Here we ignore reflection and assume perfect transmission. Though an approximation, it is a reasonable one: lenses are generally manufactured with coatings designed to reduce the reflection to around of the radiance carried by the ray. (However, modeling this small amount of reflection can be important for capturing effects like lens flare.)

If the ray has successfully made it out of the front lens element, it just needs to be transformed from lens space to camera space.

`rLens`from lens system space back to camera space>>=

The `TraceLensesFromScene()` method is quite similar to
`TraceLensesFromFilm()` and isn’t included here. The main differences
are that it traverses the elements from front-to-back rather than
back-to-front. Note that it assumes that the ray passed to it is already
in camera space; the caller is responsible for performing the
transformation if the ray is starting from world space. The returned ray
is in camera space, leaving the rear lens element toward the film.

### 6.4.3 The Thick Lens Approximation

The thin lens approximation used in Section 6.2.3 was based on the simplifying assumption that the lens system had 0 thickness along the optical axis. The thick lens approximation of a lens system is slightly more accurate in that it accounts for the lens system’s extent. After introducing the basic concepts of the thick lenses here, we’ll use the thick lens approximation to determine how far to place the lens system from the film in order to focus at the desired focal depth in Section 6.4.4.

The thick lens approximation represents a lens system by two pairs of
distances along the optical axis—the *focal points* and the depths of the
*principal planes*; these are two of the *cardinal points* of a
lens system. If rays parallel to the optical axis are traced through an
ideal lens system, all of the rays will intersect the optical axis at the
same point—this is the focal point. (In practice, real lens systems aren’t
perfectly ideal and incident rays at different heights will intersect the
optical axis along a small range of values—this is the *spherical
aberration*.) Given a specific lens system, we can trace rays parallel to
the optical axis through it from each side and compute their intersections
with the axis to find the focal points. (See
Figure 6.20.)

`lenses/dgauss.dat`with an incident ray from the scene parallel to the optical axis (above the axis), and a ray from the film parallel to the optical axis (below). The intersections with the optical axis of the rays leaving the lens system due to these incident rays give the two focal points, (on the film side) and (on the scene side). The principal planes and are given by the intersection of the extension of each pair of incident and exiting rays with the original rays and are shown here as blue lines perpendicular to the optical axis.

Each principal plane is found by extending the incident ray parallel to the optical axis and the ray leaving the lens until they intersect; the depth of the intersection gives the depth of the corresponding principal plane. Figure 6.20 shows a lens system with its focal points and and principal planes at values and . (As in Section 6.2.3, primed variables represent points on the film side of the lens system, and unprimed variables represent points in the scene being imaged.)

Given the ray leaving the lens, finding the focal point requires first computing the value where the ray’s and components are zero. If the entering ray was offset from the optical axis only along , then we’d like to find such that . Thus,

In a similar manner, to find the for the principal plane where the ray leaving the lens has the same height as the original ray, we have , and so

Once these two values have been computed, the ray equation can be used to find the coordinates of the corresponding points.

The `ComputeCardinalPoints()` method computes the depths of the
focal point and the principal plane for the given rays. Note that it
assumes that the rays are in camera space but returns values along the
optical axis in lens space.

The `ComputeThickLensApproximation()` method computes both pairs of
cardinal points for the lens system.

First, we must choose a height along the axis for the rays to be traced. It should be far enough from so that there is sufficient numeric precision to accurately compute where rays leaving the lens system intersect the axis, but not so high up the axis that it hits the aperture stop on the ray through the lens system. Here, we use a small fraction of the film’s diagonal extent; this works well unless the aperture stop is extremely small.

To construct the ray from the scene entering the lens system `rScene`,
we offset a bit from the front of the lens. (Recall that the ray passed to
`TraceLensesFromScene()` should be in camera space.)

An equivalent process starting from the film side of the lens system gives us the other two cardinal points.

### 6.4.4 Focusing

Lens systems can be focused at a given depth in the scene by moving the system in relation to the film so that a point at the desired focus depth images to a point on the film plane. The Gaussian lens equation, (6.3), gives us a relation that we can solve to focus a thick lens.

For thick lenses, the Gaussian lens equation relates distances from a point in the scene at and the point it focuses to by

For thin lenses, , and Equation (6.1) follows.

If we know the positions and of the principal planes and the focal length of the lens and would like to focus at some depth along the optical axis, then we need to determine how far to translate the system so that

The focal point on the film side should be at the film, so , and , the given focus depth. The only unknown is , and some algebraic manipulation gives us

(There are actually two solutions, but this one, which is the closer of the two, gives a small adjustment to the lens position and is thus the appropriate one.)

`FocusThickLens()` focuses the lens system using this approximation.
After computing , it returns the offset along the axis from the
film where the lens system should be placed.

`delta`, to focus at

`focusDistance`>>

Equation (6.4) gives the offset . The focal length of the lens is the distance between the cardinal points and . Note also that the negation of the focus distance is used for , since the optical axis points along negative .

`delta`, to focus at

`focusDistance`>>=

We can now finally implement the fragment in the `RealisticCamera`
constructor that focuses the lens system. (Recall that the thickness of
the rearmost element interface is the distance from the interface to the
film.)

### 6.4.5 The Exit Pupil

From a given point on the film plane, not all rays toward the rear lens element
will successfully exit the lens system; some will be blocked by the
aperture stop or will intersect the lens system enclosure. In turn, not
all points on the rear lens element transmit radiance to the point on the
film. The set of points on the rear element that do carry light through
the lens system is called the *exit pupil*; its size and position
vary across viewpoints on the film plane. (Analogously, the entrance
pupil is the area over the front lens element where rays from a given point
in the scene will reach the film.)

Figure 6.21 shows the exit pupil as seen from two points on the film plane with a wide angle lens. The exit pupil gets smaller for points toward the edges of the film. An implication of this shrinkage is vignetting.

When tracing rays starting from the film, we’d like to avoid tracing too many rays that don’t make it through the lens system; therefore, it’s worth limiting sampling to the exit pupil itself and a small area around it rather than, for example, wastefully sampling the entire area of the rear lens element.

Computing the exit pupil at each point on the film plane before tracing a
ray would be prohibitively expensive; instead the `RealisticCamera`
implementation precomputes exit pupil bounds along segments of a line on
the film plane. Since we assumed that the lens system is radially
symmetric around the optical axis, exit pupil bounds will also be radially
symmetric, and bounds for arbitrary points on the film plane can be found by
rotating these segment bounds appropriately
(Figure 6.22). These bounds are then used
to efficiently find exit pupil bounds for specific film sample positions.

`RealisticCamera`computes bounds of the exit pupil at a series of segments along the axis of the film plane, up to the distance from the center of the film to a corner. (b) Due to the assumption of radial symmetry, we can find exit pupil bounds for an arbitrary point on the film (solid dot) by computing the angle between the point and the axis. If a point is sampled in the original exit pupil bounds and is then rotated by , we have a point in the exit pupil bounds at the original point.

One important subtlety to be aware of is that because the lens system is focused by translating it along the optical axis, the shape and position of the exit pupil change when the focus of the lens system is adjusted. Therefore, it’s critical that these bounds be computed after focusing.

The `BoundExitPupil()` method computes a 2D bounding box of the exit
pupil as seen from a point along a segment on the film plane. The bounding
box is computed by attempting to trace rays through the lens system at a
set of points on a plane tangent to the rear lens element. The bounding
box of the rays that make it through the lens system gives an approximate
bound on the exit pupil—see
Figure 6.23.

`BoundExitPupil()`takes an interval along the axis on the film. It samples a series of points along the interval (bottom of the figure). For each point, it also samples a point on the bounding box of the rear lens element’s extent on the plane tangent to its rear. It computes the bounding box on the tangent plane of all of the rays that make it through the lens system from points along the interval.

The implementation samples the exit pupil fairly densely—at a total of points for each segment. We’ve found this sampling rate to provide good exit pupil bounds in practice.

The bounding box of the rear element in the plane perpendicular to it is not enough to be a conservative bound of the projection of the exit pupil on that plane; because the element is generally curved, rays that pass through the plane outside of that bound may themselves intersect the valid extent of the rear lens element. Rather than compute a precise bound, we’ll increase the bounds substantially. The result is that many of the samples taken to compute the exit pupil bound will be wasted; in practice, this is a minor price to pay, as these samples are generally quickly terminated during the lens ray-tracing phase.

The sample point on the film is found by linearly interpolating between
the interval endpoints. The `RadicalInverse()` function that is
used to compute the interpolation offsets for the sample point inside the
exit pupil bounding box will be defined later, in
Section 7.4.1. There, we will see that the sampling
strategy implemented here corresponds to using Hammersley points in 3D; the
resulting point set minimizes gaps in the coverage of the overall 3D
domain, which in turn ensures an accurate exit pupil bound estimate.

Now we can construct a ray from `pFilm` to `pRear` and determine
if it is within the exit pupil by seeing if it makes it out of the front of
the lens system. If so, the exit pupil bounds are expanded to include this
point. If the sampled point is already inside the exit pupil’s bounding box
as computed so far, then we can skip the lens ray tracing step to save a
bit of unnecessary work.

It may be that none of the sample rays makes it through the lens system;
this case can legitimately happen with some very wide-angle lenses where
the exit pupil vanishes at the edges of the film extent, for example. In
this case, the bound doesn’t matter and `BoundExitPupil()` returns the
bound that encompasses the entire rear lens element.

While one sample may have made it through the lens system and one of its neighboring samples didn’t, it may well be that another sample very close to the neighbor actually would have made it out. Therefore, the final bound is expanded by roughly the spacing between samples in each direction in order to account for this uncertainty.

Given the precomputed bounds stored in
`RealisticCamera::exitPupilBounds`, the `SampleExitPupil()` method
can fairly efficiently find the bounds on the exit pupil for a given point
on the film plane. It then samples a point inside this bounding box for
the ray from the film to pass through. In order to accurately model the
radiometry of image formation, the following code will need to know the area of
this bounding box, so it is returned via `sampleBoundsArea`.

`pFilm`with axis>> }

Given the pupil’s bounding box, a point inside it is sampled via linear
interpolation with the provided `lensSample` value, which is in
.

Because the exit pupil bound was computed from a point on the film along
the axis but the point `pFilm` is an arbitrary point on the film,
the sample point in the exit pupil bound must be rotated by the same
angle as `pFilm` makes with the axis.

`pFilm`with axis>>=

### 6.4.6 Generating Rays

Now that we have the machinery to trace rays through lens systems and to
sample points in the exit pupil bound from points on the film plane,
transforming a `CameraSample` into a ray leaving the camera is fairly
straightforward: we need to compute the sample’s position on the film plane
and generate a ray from this point to the rear lens element, which is then
traced through the lens system.

`pFilm`, corresponding to

`sample.pFilm`>>

`pFilm`through lens system>>

`RealisticCamera`ray>> <<Return weighting for

`RealisticCamera`ray>>

The `CameraSample::pFilm` value is with respect to the overall
resolution of the image in pixels. Here, we’re operating with a physical
model of a sensor, so we start by converting back to a sample in .
Next, the corresponding point on the film is found by linearly interpolating
with this sample value over its area.

`pFilm`, corresponding to

`sample.pFilm`>>=

`SampleExitPupil()` then gives us a point on the plane tangent to the
rear lens element, which in turn lets us determine the ray’s direction. In
turn, we can trace this ray through the lens system. If the ray is blocked
by the aperture stop or otherwise doesn’t make it through the lens system,
`GenerateRay()` returns a 0 weight. (Callers should be sure to
check for this case.)

`pFilm`through lens system>>=

If the ray does successfully exit the lens system, then the usual details have to be handled to finish its initialization.

`RealisticCamera`ray>>=

The fragment <<Return weighting for `RealisticCamera` ray>>
will be defined later, in Section 13.6.6, after some
necessary background from Monte Carlo integration has been introduced.

### 6.4.7 The Camera Measurement Equation

Given this more accurate simulation of the process of real image formation, it’s also worthwhile to more carefully define the radiometry of the measurement made by a film or a camera sensor. Rays from the exit pupil to the film carry radiance from the scene; as considered from a point on the film plane, there is thus a set of directions from which radiance is incident. The distribution of radiance leaving the exit pupil is affected by the amount of defocus blur seen by the point on the film—Figure 6.24 shows two renderings of the exit pupil’s radiance as seen from two points on the film.

Given the incident radiance function, we can define the irradiance at a point on the film plane. If we start with the definition of irradiance in terms of radiance, Equation (5.4), we can then convert from an integral over solid angle to an integral over area (in this case, an area of the plane tangent to the rear lens element that bounds the exit pupil, ) using Equation (5.6). This gives us the irradiance for a point on the film plane:

Figure 6.25 shows the geometry of the situation.

`RealisticCamera::LensRearZ()`, and is the angle between the vector from to and the optical axis.

Because the film plane is perpendicular to the exit pupil plane, . We can further take advantage of the fact that the distance between and is equal to the axial distance from the film plane to the exit pupil (which we’ll denote here by ) divided by . Putting this all together, we have

For cameras where the extent of the film is relatively large with respect to the distance , the term can meaningfully reduce the incident irradiance—this factor also contributes to vignetting. Most modern digital cameras correct for this effect with preset correction factors that increase pixel values toward the edges of the sensor.

Integrating irradiance at a point on the film over the time that the
shutter is open gives *fluence*, which is the radiometric unit for
energy per area, .

Measuring fluence at a point captures the effect that the amount of energy received on the film plane is partially related to the length of time the camera shutter is open.

Photographic film (or CCD or CMOS sensors in digital cameras) actually measure radiant energy over a small area. Taking Equation (6.6) and also integrating over sensor pixel area, , we have

the Joules arriving at a pixel.

In Section 13.2, we’ll see how Monte Carlo can be applied
to estimate the values of these various integrals. Then in
Section 13.6.6 we will define the fragment <<Return
weighting for `RealisticCamera` ray>> in
`RealisticCamera::GenerateRay()`; various approaches to computing the
weight allow us to compute each of these quantities.
Section 16.1.1 defines the *importance function*
of a camera model, which characterizes its sensitivity to incident
illumination arriving along different rays.