Importance Sampling for Production Rendering Mark Colbert
Google (formerly Disney’s ImageMovers Digital)
Simon Premoze Guillaume Francois
Weta Digital (formerly Moving Picture Company)
Problem
Problem • Focus on efficiently evaluating the illumination integral
Problem • Focus on efficiently evaluating the illumination integral
• Looking predominately at direct lighting
Illumination Integral
Illumination Integral
Illumination Integral
Illumination Integral
Illumination Integral
Illumination Integral
Illumination Integral ωo ωi
Lo (ωo , x) =
!
Ω
Li (ωi , x)f (ωo , ωi ) cos θi dωi
Illumination Integral •
Li - incoming light
ωo Li ωi
Lo (ωo , x) =
!
Ω
Li (ωi , x)f (ωo , ωi ) cos θi dωi
Illumination Integral ωo Li ωi
• •
Li - incoming light f - Bidirectional Reflectance Distribution Function (BRDF)
f
Lo (ωo , x) =
!
Ω
Li (ωi , x)f (ωo , ωi ) cos θi dωi
Illumination Integral ωo Li ωi
• •
Li - incoming light
•
Cosine for projection
f - Bidirectional Reflectance Distribution Function (BRDF)
f
Lo (ωo , x) =
!
Ω
Li (ωi , x)f (ωo , ωi ) cos θi dωi
Illumination Integral
Illumination Integral • Evaluate and sum all incoming directions
Illumination Integral • Evaluate and sum all incoming directions
• Computationally Expensive
Importance Sampling • Re-orient the sample directions to point in directions that provide better estimates with fewer samples
Importance Sampling • Re-orient the sample directions to point in directions that provide better estimates with fewer samples
Overview • BRDF Importance Sampling • Filtered Importance Sampling (FIS) • FIS for Area Light • Multiple Importance Sampling Sampling at MPC • Importance Guillaume Francois Simon Premoze
Goal • With the presentation and course notes,
the goal is to provide you with the tools to implement importance sampling in your production pipeline
Integration • Discretize integral into smaller solid angles N ! 2π ≈ Li (ωi , x)f (ωi , ωo ) cos θi N i=1 2
Integration • Deterministic Sampling • Break into even sized pieces (e.g. Trapezoidal Rule)
• Stochastic Sampling • Randomly choose sample directions • Monte Carlo Integration
Importance Sampling •
Sample according to a Probability Density Function (PDF)
• •
Effectively an educated guess
•
PDF
Map from a uniform sample space to a nonuniform space
• • •
Peaks represent important regions Non-negative Integrates to one
Importance Sampling • Need to adjust for not evenly spacing the samples
Importance Sampling • Need to adjust for not evenly spacing the samples
Importance Sampling • Need to adjust for not evenly spacing the samples
Importance Sampling • Need to adjust for not evenly spacing the samples
Importance Sampling • Need to adjust for not evenly spacing the samples
Importance Sampling • Need to adjust for not evenly spacing the samples
N ! 1 Li (ωi , x)f (ωi , ωo ) cos θi ≈ N i=1 p(ωi )
What do we sample from?
What do we sample from? • BRDF
What do we sample from? • BRDF • Environment
What do we sample from? • BRDF • Environment • Both?
BRDF Sampling • Sampling depends on choice of BRDF • Common BRDFs [1982] • Cook-Torrance (similar to specular in RenderMan) • Phong [1975]
Phong BRDF • Exponentiated cosine lobe around the specular reflection direction
!
n
Li (ωi , x) cos θs cos θi dωi
Ω
θs
Phong Sampling • Convert to spherical coordinates centered around the specular direction
p(θs , φs ) ∝ cosn θs sin θs
• Normalize cosn θs sin θs
p(θs , φs ) = ! π/2 ! 2π 0
0
cosn θs sin θs dφs dθs
(n + 1) n cos θs sin θs = 2π
Inverse Transform Sampling • Discrete case
S1
S2
S3
S4
S5
Inverse Transform Sampling • Stack the transforms on top of each other and sample uniformly
Inverse Transform Sampling • Stack the transforms on top of each other and sample uniformly
S1
S2
S3
S4
S5
Inverse Transform Sampling • Stack the transforms on top of each other Uniform Sampling
and sample uniformly
S1
S2
S3
S4
S5
Inverse Transform Sampling • Stack the transforms on top of each other Uniform Sampling
and sample uniformly
S1
S2
S3
S4
S5
Inverse Transform Sampling • Analytical equivalent is to integrate over the PDF
P (Θ) =
!
Θ
p(θ)dθ
0
• P is called
Cumulative Distribution Function (CDF)
• Invert and solve for the sample direction P (Θ) = ξ P −1 (ξ) = Θ
Phong Sampling • Often better to sample over each
dimension (depends on sampling pattern)
• Two dimensional equation • Create marginal and conditional
probabilities for each dimension and sample independently
Marginal Density • Integrate over entire domain of a dimension
Marginal Density • Integrate over entire domain of a dimension
Marginal Density • Integrate over entire domain of a dimension
Marginal Density • Integrate over entire domain of a dimension
Marginal Density • Integrate over entire domain of a dimension
p(θs ) =
!
2π
p(θs , φs )dφs
0
= (n + 1) cos θs sin θs n
Conditional Density • Using Bayes law, we can find the conditional distribution
Conditional Density • Using Bayes law, we can find the conditional distribution
Conditional Density • Using Bayes law, we can find the conditional distribution
Conditional Density • Using Bayes law, we can find the conditional distribution
p(θs , φs ) p(φs |θs ) = p(θs )
Conditional Density • Using Bayes law, we can find the conditional distribution
p(θs , φs ) 1 p(φs |θs ) = = p(θs ) 2π
Putting it all together • Integrate each to find the CDF • Invert to find the sample mapping: !
1 n+1
θs = arccos ξ1 φs = 2πξ2
"
Randomness • Given the mappings for importance
sampling integration we just to generate the uniformly random samples
• So what? Call rand().
Randomness
Randomness • rand() causes clumping
Randomness • rand() causes clumping • Blue-noise property:
Each sample point has a radius in which no other sample point can exist [Dunbar 2006]
Randomness • rand() causes clumping • Blue-noise property:
Each sample point has a radius in which no other sample point can exist [Dunbar 2006]
Low-Discrepancy Sequences • Quasi-random numbers (Deterministic)
• Maximal distance from each sample point (Stronger than blue noise property)
• Ideal coverage that looks non-regular to minimize visible aliasing 0,0
0,0
1,1
1,1
Folded Hammersley Sequence (ξ1 , ξ2 ) =
!
" i + 0.5 , Φ2 (i) N
Φ2 (i) ! 1 2 3 4
Binary! 1 10 11 100
Inverse Binary 0.1 0.01 0.11 0.001
Value 0.5 0.25 0.75 0.125
Let’s see some pictures... • Using importance sampling we can quickly produce okay results
40 Samples Per Pixel
Let’s see some pictures... • Using importance sampling we can quickly produce okay results
40 Samples Per Pixel
Deterministic Sampling
• Replace noise with aliasing
40 Samples Per Pixel
Deterministic Sampling
• Replace noise with aliasing
40 Samples Per Pixel
Deterministic Sampling
• Replace noise with aliasing
40 Samples Per Pixel
Filtered Importance Sampling
• Replace noise/aliasing with a smooth result
40 Samples Per Pixel
Filtered Importance Sampling
• Replace noise/aliasing with a smooth result
40 Samples Per Pixel
Filtered Importance Sampling
• Replace noise/aliasing with a smooth result
40 Samples Per Pixel
FIS Foundation • Warp the environment proportionally to the BRDF
FIS Foundation • Warp the environment proportionally to the BRDF
FIS Foundation • Warp the environment proportionally to the BRDF
FIS Foundation • Observing the environment proportional to the BRDF we have evenly spaced samples
Environment ! BRDF ! cos !
FIS Foundation • Observing the environment proportional to the BRDF we have evenly spaced samples
Environment ! BRDF ! cos !
FIS Foundation • Undersampling causes aliasing
FIS Foundation • Suppress the aliasing by pre-filtering
FIS Foundation
FIS Foundation • Ideal filter region
defined by going between the neighboring points
FIS Foundation • Ideal filter region
defined by going between the neighboring points
FIS Foundation • Ideal filter region
defined by going between the neighboring points
• Gives the solid angle: 1 Ωs = N · p(ωi )
FIS with Environment Maps • Must convert solid angle to a filter region in the image-based environment
• MIP-Map Filtering • Ptex Filtering (RenderMan)
MIP-Map Filtering • Pyramid of filtered images,
where each level is 1/4 the size of the previous
• Commonly available in all
renderers (including GPUs)
MIP-Map Filtering • Convert the solid angle Ωp = d(ωi ) to pixels for some mapping
• Use ratio of solid angle for a pixel to the total solid angle !
1 Ωs l = max log2 ,0 2 Ωp
"
w·h
MIP-Map Filtering • Which parameterization is ideal? • Most filtering engines treat each map of an environment independently
• Need a mapping that has overlap between other maps
MIP-Map Filtering • One solution, scaled dual paraboloid
Can we do better?
Can we do better? • •
YES! Ptex-based filtering [Burley 2008]
•
RenderMan
Cube map that filters across edges of faces giving seamless filters
• •
Use environment call Must convert solid angle into blur parameter:
blur =
!
3 Ωs 2π
Area Light • Environment lighting still a relatively newer construct for production
• Soft lighting for female characters in ACC
IMD’s Area Light • • • •
Decoupled model (wrong, but controllable) Visibility
•
Done independently and used to attenuate other components
Diffuse
•
Used analytical form factor computation [Baum et al. 1989]
Glossy reflections
•
Filtered Importance Sampling
Area Light • Still importance sample the BRDF • Send out samples to intersect area light rectangle
• Augment the rays with ray differentials to
project the filter region onto the area light
Spherical Cap • How wide do we make the ray differentials? Ωs !
Ωs θs = arccos 1 − 2π
"
θs
Ray differentials • Ideally 2 rays to capture the shape of the projected filter region
• Requires orthogonal basis vectors to the sample direction
• This requires a smooth coordinate frame
Area Light Filtering • •
Intersect
• •
Axis aligned clipping
Compute the area around the central intersection ...and filter!
Zi
Zi (a)
Zi (b)
(c)
Area Filtering
Area Light
Camera
MIP-Map FIS Results
n = 17 100 Samples
n=3 5 Samples
Environment Sampling
Reference
Filtered Sampling
MIP-Map FIS Results • Extreme anisotropy does not behave as well as isotropic BRDFs
"x = 0.01 "y = 0.01
"x = 0.01 "y = 0.08 Ward BRDF
"x = 0.01 "y = 0.29
MIP-Map FIS Results
50 Samples
200 Samples
Reference
50 Samples
5 Samples
Summary • Importance Sampling provides a good framework for numerically solving Illumination Integral
• FIS provides a production proven method of integrating glossy surface reflection
• FIS replaces alias or noise with smooth reflections ideal for production scenes
Acknowledgements • Google • ImageMovers Digital • Doug Epps • Davy Wentworth • Christophe Hery • Jaroslav Krivanek