INFORMATIK
Online Accelerated Rendering of Visual Hulls in Real Scenes February 6th, 2003
Ming -Peter Seidel Ming Li, Li, Marcus Marcus Magnor Magnor,, Hans Hans-Peter Seidel Max Max-Planck Planck--Institute for Max-Planck-Institute for Computer Computer Science Science Saarbruecken Saarbruecken, Saarbruecken,, Germany Germany
WSCG‘03
Ming Li
Talk Outline
INFORMATIK
Motivation System overview Visual hull reconstruction Rendering improvements Performance & Results Conclusions & Future work WSCG‘03
Ming Li
Introduction
INFORMATIK
What do we want? •• Acquire -world object Acquire dynamic dynamic 3D 3D geometry geometry of of real real-world object on -line and -time on-line and render render itit in in its its environment environment in in real real-time –– 3D 3D interactive interactive TV, TV, computer computer games, games, tele tele-communication communication and and sports sports analysis analysis etc. etc.
Visual hull •• An An approximate approximate shell shell that that envelopes envelopes the the true true geometry geometry of of the the object object •• Suitable Suitable for for real real time time reconstruction reconstruction and and rendering rendering
WSCG‘03
Ming Li
Motivation
INFORMATIK
Our work •• Develop Develop improved improved rendering rendering algorithms algorithms accelerated accelerated by by graphics graphics hardware hardware •• Background Background rendering rendering –– Noticeable Noticeable boundary boundary seams seams between between multiple multiple views views •• Visual Visual hull hull rendering rendering –– Texture Texture will will project project through through
WSCG‘03
Ming Li
System Overview •• ••
INFORMATIK
Clients Clients — — Three Three AMD AMD Athlon Athlon 1.1GHz 1.1GHz machines machines Server -processor machine Server — — Intel Intel P4 P4 1.7GHz 1.7GHz dual dual-processor machine with with GeForce GeForce 33 graphics graphics card card
•• Six Six cameras cameras are are arranged arranged roughly roughly along along aa circle circle •• Pre -calibrated using Pre-calibrated using Tsai’s Tsai’s algorithm algorithm •• Sync Sync via via external external trigger trigger •• Image Image resolution resolution — — 320x240 320x240 pixels pixels WSCG‘03
Ming Li
Visual Hull Reconstruction
INFORMATIK
Silhouette extraction •• Static Static background background acquired acquired beforehand beforehand •• Background Background differencing differencing &Polygon &Polygon approximation approximation
Polyhedral visual hull reconstruction •• Project Project silhouette silhouette polygons polygons back back to to 3D 3D space space •• 3D 3D intersection intersection WSCG‘03
Ming Li
Background Blending
INFORMATIK
Basic approach •• Project Project multiple multiple views views onto onto aa background background geometry geometry proxy proxy N 1 •• Target Target blending blending function: function: C = N * ∑ C k
Problem
k =1
•• Need Need per-pixel per-pixel accumulation accumulation and and normalization normalization
Solution •• Frame Frame buffer buffer blending blending for for accumulation accumulation •• Alpha Alpha channel channel for for pre-normalization pre-normalization WSCG‘03
Ming Li
Background Blending Details
INFORMATIK
•• Multi-pass Multi-pass rendering rendering and and accumulate accumulate blended blended results results in in frame frame buffer buffer •• Initial Initial frame frame buffer buffer (0, (0, 0, 0, 0, 0, 1), 1), SRC_ALPHA=0.5 SRC_ALPHA=0.5 •• Color Color channel channel blending blending C = C s * DST _ALPHA + C d * (1 − DST _ALPHA )
–– This This formula formula computes computes average average color color IfIf DST_ALPHA’s DST_ALPHA’s form form reciprocal reciprocal number number series series •• Alpha Alpha channel channel blending blending A = As * 0 .12 + Ad * SRC _ ALPHA
–– Approximate Approximate reciprocal reciprocal number number series series WSCG‘03
Ming Li
Background Blending Results
Without Blending
WSCG‘03
INFORMATIK
With Blending
Ming Li
Projective Texture Mapping & Shadow Mapping
INFORMATIK
Normal projective texture mapping •• Occluded Occluded parts parts also also get get textured textured
Shadow mapping •• Render - of-view Render from from the the light’s light’s point point-of-view to to generate generate depth depth map map •• Render -of-view Render from from the the eye’s eye’s point point-of-view –– Use Use depth depth map map as as aa projective projective texture texture •• Produce Produce texel texel 11 for for lit lit parts parts ,, 00 for for shadowed shadowed parts parts WSCG‘03
light Zd in depth map
eye
Z position relative to light Ming Li
Opaque Projective Texture Mapping
INFORMATIK
Opaque projective texture mapping •• Multiple Multiple textures textures –– Tex0: Tex0: Color Color texel texel,, Tex1: Tex1: Shadow Shadow texel texel –– Final Final fragment: fragment: Tex0*Tex1 Tex0*Tex1 •• Enable Enable alpha alpha testing testing to to remove remove occluded occluded parts parts
Problem
Shadow leak
•• Shadow Shadow leak leak due due to to insufficient insufficient sampling sampling of of depth depth map map WSCG‘03
Ming Li
Extended Opaque Projective Texture Mapping
INFORMATIK
Two options •• Increase Increase depth depth map map resolution resolution –– Cost Cost more more rendering rendering time time –– Consume Consume more more texture texture memory memory •• Render Render aa fatter fatter version version of of depth depth map map –– 22-pass -pass depth depth map map rendering rendering 8 811stst Pass: Pass: Line Line mode mode using using thick thick line line width width nd Pass: 8 822nd Pass: Normal Normal mode mode Shadow leak artifacts are completely removed WSCG‘03
Ming Li
Extended Opaque Projective Texture Mapping
Texture Texture-though Texture-though
WSCG‘03
Remove texture -through Remove texturetexture-through
INFORMATIK
Remove Remove Shadow Shadow leak leak
Ming Li
Rendering Results
INFORMATIK
Performance measurement •• •• •• ••
Triangle Triangle number number of of dynamic dynamic object object — — 400 400 ~~ 500 500 Texture -37% Texture packing packing improves improves rendering rendering speed speed by by 20 20-37% 15 15 fps fps for for visual visual hull hull reconstruction reconstruction 23 23 fps fps for for textured textured visual visual hull hull rendering rendering with with background background
WSCG‘03
Ming Li
Video
WSCG‘03
INFORMATIK
Ming Li
Conclusions & Future Work
INFORMATIK
Conclusions •• ••
Introduced Introduced aa novel novel blending blending algorithm algorithm to to hide hide boundary boundary seams seams for for background background rendering rendering Extended Extended opaque opaque projective projective texture texture mapping mapping to to handle handle shadow shadow leak leak artifacts artifacts
Future Work ••
WSCG‘03
Better Better and and faster faster rendering rendering by by exploiting exploiting flexible flexible programmability -generation graphics programmability of of next next-generation graphics hardware hardware Ming Li
