Non-Photorealistic Rendering in Augmented Reality Environments
Description
In modern computer graphics, most research effort has been put into the creation of photorealistic images, which resemble the real world as much as possible. Nowadays, it is often impossible to distinguish between a real-world image and a digital creation. In contrast to this, the field of non-photorealistic computer graphics is comparatively young.
One of its goals is to digitally reproduce the creative techniques
of classic art forms. The goal of this project was to
combine Augmented Reality technology with NPR techniques. Our framework is based on ARToolKit and includes actually three rendering techniques:
Painterly renderer
Toon shader
Gooch shader
Painterly renderer
A method for rendering animation in a painterly style, which was first presented in [Meier, B. J.: Painterly Rendering
for
Animation. In: Proceedings of SIGGRAPH 96, p. 477-484, 1996] , is the basis of this shader. While previous
algorithms only worked for still
frames, the new method successfully extends them to be applicable in animations.
This is achieved mainly by two techniques: organizing
brush strokes in particle systems and the use of reference
pictures to compute their properties. The result is a frame-to-frame coherence,
which is a requirement of animation.
Modern 3D hardware should make it possible to achieve this effect in real-time applications. Proving
this statement is the main goal of this thesis. The possibility to send instructions to the graphics card and compute
them either per vertex
or per pixel allows both the rendering of thousands of brush strokes per frame,
and the
correct application of their properties.
Programming the graphics hardware is done with Cg, a high level shading
language. Fast access to reference pictures is achieved by using
OpenGL extensions that allow direct rendering
to textures. A closer description of this algorithm including some downloads can be found Daniel's website.
Daniel implemented a very cool framework (called NPAR) which was the base for the second prototype, called Vincent.
Vincent is a NPR framework that combines ARToolKit (marker detection framework), Cg, and other
nice loaders
(e.g. Lib3DS, RenderTexture, etc.).
Toon shader
The objects are rendered with a constant, sharply delineated
color with an outline of the object. In our case
we replace diffuse and specular lighting colors with a two-valued step function, where
the values are calculated by
using a 1D texture as a lookup table.
Gooch shader
Instead of using Phong shading, Gooch involves a so-called
warm/cold factor into the
formula. Consequently, we get the following formula:
I = ((1+l*n)/2)*Kcool + (1- ((1+l*n)/2)*kwarm
Publications
Photorealism or/and Non-Photorealism in Augmented Reality , Haller M., In ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry, VRCAI 2004, pp. 189- 196, Singapore.
Real-Time Painterly Rendering for MR Applications , Haller M., Sperl D., In International
Conference on Computer Graphics and Interactive Techniques in Australasia and South
East Asia, Graphite 2004, pp. 30 - 38, Singapore.
Künstlerisches Rendering für Echtzeit-Applikationen, Sperl D., Master Thesis, Upper Austria University of Applied Sciences, Media Technology and Design, Hagenberg, 2003.
Additional Data
The original algorithm of Barbara Meier has been modified to achieve real-time performance. The first figure shows the modified pipeline.
Nice results of NPAR: Different base shaders results in different final painterly renderer results.
Some
results of Vincent: Isn't it cool to see the "Bedroom in Arles" in 3D?
Additional results of Vincent: Which renderer should be used in AR? Very nice results are achieved by using the Gooch renderer (last two figures)
Download the videos related to Vincent. Using ARToolkit and an AR environment is a really stressful testing environment for your algorithm!
