Research Abstracts - 2007

Introduction

Many tasks in image processing and machine vision take the form of image to image mapping. Given an image, X, the problem is to estimate another image Y, which is in register with the first. Classical image processing problems such as denoising and superresolution are examples of image mapping. Within machine vision, estimating intrinsic images [1] such as albedo and illumination, or shape, or optic flow would be other examples. Stylistic image mapping, under the names "texture transfer" [2] and "image analogies" [3], forms another category.

We describe a set of techniques for mapping one image to another based on the statistics of a training set. We apply these techniques to the problems of image denoising and superresolution, but they should also be useful for other vision problems where training data are available. Given a local feature vector computed from an input image patch, we learn to estimate a subband coefficient of the output image conditioned on the patch. This entails approximating a multidimensional function, which we make tractable by nested binning and linear regression within bins. This method performs as well as nearest neighbor techniques, but is much faster. After attaining this local (patch based) estimate, we force the marginal subband histograms to match a set of target histograms, in the style of Heeger and Bergen [4]. The target histograms are themselves estimated from the image data, based on training. With the combined techniques, denoising performance is similar to state of the art techniques in terms of PSNR, and is slightly superior in subjective quality. In the case of superresolution, our techniques produce much higher subjective quality than the competing methods, allowing us to attain very large increases in apparent resolution. We have verified this advantage with user studies.

Results

Denoising and superresolution results are shown in Figure 1 and Figure 2.

Figure 1. Denoising results. First image: noisy image, with Gaussian additive noise. Second image: our denoised image.

Figure 2. Superresolution results. A 311x258 image is downsampled by 4 in each direction, and then super resolved. First image: bicubic upsampling. Second image: our superresolution result.

References:

[1] H. Barrow and J. Tenenbaum. Recovering intrinsic scene characteristics from images. Computer Vision Systems, 1978

[2] A. Efros and W.T. Freeman. Image quilting for texture synthesis and transfer. In Siggraph, pages 341-346, 2001.

[3] A. Hertzmann, C.E.Jacobs, N. Oliver, B. Curless, and D.H. Salesin. Image analogies. In Siggraph, pages 327-340, 2001

[4] D.J. Heeger and J.R. Bergen. Pyramid-based texture analysis/synthesis. In Siggraph, 1995