Research Abstracts - 2007

Texture Transfer Using Geometry Correlation

Tom Mertens, Jan Kautz, Jiawen Chen, Philippe Bekaert & Frédo Durand

Introduction

The visual richness of textures has long been recognized as critical to photorealism. Recent advances have enabled the synthesis of elaborate textures from example. However, for a large class of objects, the texture can be quite simple locally, but may exhibit rich variation over the surface. For example, this is the case for weathered objects where dirt or corrosion tends to manifest itself differently over the surface, but it can also be caused by other aspects of an artifact's fabrication and history. In addition, texture variation is often correlated with local geometric characteristics. Instead of modeling the underlying processes that produce a particular texture variation, we empirically model the variation with statistical correlation. We present a technique to transfer such geometry-dependent texture variation from an example textured model to new geometry in a visually consistent way. Our goal is not to simulate the physics of weathering processes but to reproduce the rich visual appearance of a textured object.

Approach

Given a source 3D mesh A, its corresponding texture map A', and a target 3D mesh B, our algorithm synthesizes a texture map B' that captures the same geometry-dependent texture variation as the source model (A : A' :: B : B'). Our method begins by computing an over-complete set of geometric features such as curvature and visibility, which we correlate with the texture color using Canonical Correlation Analysis (CCA). CCA gives us a low-dimensional subspace of features which we use as a guidance field to drive constrained texture synthesis.

To synthesize structured textures, we extend the Image Analogies technique by Hertzmann et al. [1] to operate over 3D surfaces. For unstructured textures that exhibit strong geometric correlation that may extend over large spatial areas, we introduce a novel constrained parametric texture synthesis technique based on the Heeger and Bergen model [2]. On a Pentium 4 2.8 GHz workstation, our algorithm takes roughly 2 hours to transfer a structured texture, and only 15 minutes for an unstructured texture.

Conclusion

This work introduces a novel method for transferring geometry-dependent texture variation from one model to another. It presents two constrained texture synthesis approaches, with a tradeoff between being able to handle structure versus preserving strong geometric correlation. For future work, we would like to apply the technique to higher-order material appearance models such as BRDFs or BTFs.

This work has been published at the Eurographics Symposium on Rendering [3]. More details are available at the project page.

Acknowledgements

We would like to thank the following people for their help: Jon Chu, Eugene Hsu, Hendrik Lensch, Frank Van Reeth and Tiffany Wang. Bruno Lévy was so kind to provide his mesh parameterization tool. Tom Mertens received a research fellowship from the Belgian American Educational Foundation. Part of this work was realized while Tom Mertens was a PhD student at the Expertise Centre for Digital Media (Hasselt University). Jan Kautz was supported in part by an Emmy-Noether fellowship from the German Research Foundation for his stay at MIT. Philippe Bekaert received financial support from the European Regional Development Fund and the Flemish Interdisciplinary Institute for Broadband Communication. This work was supported by a National Science Foundation CAREER award 0447561 "Transient Signal Processing for Realistic Imagery", NSF Grant No. 0429739 "Parametric Analysis and Transfer of Pictorial Style", and a Microsoft Research New Faculty Fellowship held by Frédo Durand.

References:

[1] Aaron Hertzmann, Charles E. Jacobs, Nuria Oliver, Brian Curless and David H. Salesin. Image Analogies. In The Proceedings of the 28th annual conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01), pp. 327--340, Los Angeles, CA, USA, August 2001.

[2] David J. Heeger and James R. Bergen. Pyramid-based Texture Analysis/Synthesis. In The Proceedings of the 22nd annual conference on Computer Graphics and Interactive Techniques (SIGGRAPH '95), pp. 229--238, Los Angeles, CA, USA, August 1995.

[3] Tom Mertens, Jan Kautz, Jiawen Chen, Philippe Bekaert and Frédo Durand. Texture Transfer Using Geometry Correlation. In The Proceedings of the Eurographics Symposium on Rendering, pp. 273--284, Nicosia, Cyprus, June 2006.