Research Abstracts - 2006

Image Processing With Texture Power Maps

Sara L. Su, Frédo Durand & Maneesh Agrawala

Introduction

Applying selective emphasis to photographs is an important component of the visual design process. Psychophysics and computational models of attention predict that texture variation influences bottom-up attention mechanisms, yet unlike other low-level perceptual features, texture cannot be modified directly with existing image-processing software. We present a postprocessing technique that selectively modifies the salience of regions of an image. Our method modifies the spatial variation of texture using power maps, high-order features describing local frequency content in an image. Modification of power maps results in ective regional de-emphasis (through texture equalization) and emphasis (through texture sharpening, not described in this abstract). We validate our results quantitatively with a human subject search experiment and qualitatively with eye tracking data.

Power maps

First-order computational models of saliency measure the response to filter banks that extract contrast and orientation in the image. Various non-linearities can then be used to extract and combine maxima of the response to each feature. A recently-introduced second-order model performs additional image processing on the response to a first-order filter bank, effectively performing the same computation as first-order models but on what we term power maps rather than on image intensity. Higher-order features describing local frequency content, power maps have been used previously in image analysis; e.g. response to multiscale oriented filters can be used for texture discrimination. We show that power maps are also a powerful representation for manipulating frequency content in an image.

Texture equalization

We introduce an image-processing technique for selectively reducing spatial variation of texture to reduce the salience of distracting regions. In a nutshell, our texture equalization technique modifies distracting regions to make them look more like uniform textures.

We illustrate our technique with a 1D example. The input signal (Fig. 1(a)) is first band-pass filtered (b) and rectified with an absolute value non-linearity (c). (For the 2D case, we use steerable pyramid filters to compute frequency content because they permit straightforward analysis, processing, and reconstruction of images.) Pooling the rectified response by applying a low-pass filter with a Gaussian kernel captures the local frequency content. We call the resulting image the power map (d).

To reduce texture variation in the image, some portion of the high frequencies of the power maps must be removed, a seemingly trivial image-processing operation. However, we must define how a modification of the power map translates into a modification of pyramid coefficients. The exponent of the high-pass response (e) is used to scale the bandpass response. Because the goal is to reduce variation, a negative multiple of the high-pass is used as the scale factor. Note how the scaled signal (f) has been `flattened' compared to the input. In the 2D case, the scaled subbands are then recombined to produce the final texture-equalized image.

(a) Input	(b) Band-pass	(c) Rectified	(d) Power map	(e) Scale	(f) Output
Figure 1: Texture discrimination and manipulation in 1D.

Psychophysical study

To validate our technique's effectiveness, qualitative changes in user fixations on original and modified images were recorded using an eye tracker. Emphasized regions attracted and held fixations longer than de-emphasized ones. Results of a search experiment quantified the effect of our technique on response time. Subjects were asked to find a target object in a series of images, some unmodified and some in which distractors had been de-emphasized. Texture equalization resulted in a search speedup of more than 20%.

Discussion

Texture equalization is complementary to de-emphasis methods such as Gaussian blur, which increases depth-of-field effects. Reduced sharpness can be undesirable, particularly if distractors are at the same distance as the main subject. Blur removes high frequencies and emphasizes medium ones, possibly resulting in a more distracting object, while our technique makes high-frequencies more uniform, ``camouflaging'' medium-frequency content. Our technique is most effective for textured image regions, while Gaussian blur works best when small depth-of-field effects are already present and when medium-frequency content is not distracting.

(a) Original.	(b) Texture equalization.	(c) Texture sharpening.
Figure 2: In the original image, the major texture boundary exists between the angel statue and the background leaves. Texture equalization softens the boundary by making the high frequencies more uniform. Texture sharpening strengthens texture boundaries.

Acknowledgements

This material is based on work supported by the National Science Foundation under Grant No. 0429739 and the Graduate Research Fellowship Program, MIT Project Oxygen, and the Royal Dutch/Shell Group.

References:

[1] Sara L. Su, Frédo Durand, and Maneesh Agrawala. De-Emphasis of Distracting Image Regions Using Texture Power Maps. In Texture 2005: Proceedings of the 4th International Workshop on Texture Analysis and Synthesis, in conjunction with ICCV'05, pp.119-124, Beijing, China, October 2005.