Real-time Image Based Lighting in an Augmented Environment
Description
Current basic models (cf. Lambert, Blinn, Phong, etc.) simulate
the surface environment. Thus, they just focus on
local lighting conditions without considering the whole environment. Therefore,
the rendering quality
of the models achieved by these shading models is still too weak to be used convincingly
in an AR setup. Considering
an AR scene, virtual objects are facing the direct comparison with real world
objects. Consequently, it is more important
that the virtual objects have a more realistic look than they need to have in
a purely virtual scene. The
overall goal of
the project Real-time IBL is to generate a photorealistic environment,
where the virtual content
is seamlessly
integrated in the real scene.
The following steps are needed for creating an
IBL rendered AR-environment:
Capturing real-world illumination (capturing the image),
Finding the mirrored sphere in the video image,
Extracting the sphere maps (for both diffuse and specular
lighting),
Post-processing the sphere maps (e.g. blurring),
Rendering the virtual objects with image based lighting,
Finding the direction of the strongest illumination (used
as light position for casting shadows), and
Rendering soft shadows (using shadow maps).
In our case, we use the setup with a mirrored shpere in the scene
to get an illumination image of the entire scenario.
Publications
Photorealism in Augmented
Reality setups using Image Based Lighting (to be published)
Additional Data
The
most important difference of the first and the second figure
is the fact that the IBL-rendered teapot is
influenced by lighting parameters of the surrounding environment. The dark shadowing
at the left bottom of the left teapot is
not realistic and seams to be too artificially. Some results of approach are
depicted in the last
two figures.
The
first image represents the mirrored ball (Christmas ball) and
the capturing process for the reflection map. The
capturing result is depicted in the second figure. Notice that this texture can be used for the reflection.
For the diffuse lighting, we simply blur the texture using a Gaussian filter.
These three
figures depict the blurring effect using a Gaussian filter in the
fragment shader.
