Research Abstracts - 2006

Image Modeling Using Combined Texture and Motion

Joshua N. Migdal & John W. Fisher III

Figure 1: Slices of a three-dimensional seismic volume. Although an observer may have no prior knowledge about the origin or nature of these images, he or she is able to separate the images into two or more distinct regions, based on texture and perceived motion, or deformation.

Introduction

Given a set of images, we wish to model regions that are distinguishable in terms of their combined appearance and motion properties. This work is motivated by analyzing sequences of images such as those shown in Figure 1. It is clear that there are at least two very prominent and distinct regions within this image sequence. That distinction arises from a strong sense of texture orientation and scale, as well as the coherence of perceived motion, when Figure 1 is viewed as a time series. Although you could be told that the image sequence actually represents cross sections of a three dimensional seismic volume, that information is not essential for inferring models of image formation consistent with an underlying physical interpretation of the data (in fact, the circular region corresponds to a salt dome and the large, oriented bands represent layers of earth offset from the perfectly horizontal cross sections of the volume). By modeling the texture characteristics consistent with the various regions within the imagery and by modeling how they change appearance and shape with time (i.e. position along the axis normal to the image plane), we can obtain a rich interpretation of the data.

Approach

We wish to model the appearance of an image sequence as a collection of textures undergoing changes to both appearance and shape. We begin with work done by Soatto and Doretto, et. al on dynamic textures [1,2]. Dynamic textures compactly model infinitely repeatable, stationary textures, such as a flame or waves on the surface of water. However, they do not allow for changes in the appearance of the texture itself, nor do they allow for motions or other, more complicated deformations of the texture. More recent work has involved incorporating explicit, rigid models of motion within the dynamic texture framework [3,4] .

We model the deformation of a particular texture using piecewise affine transformations. Piecewise affine transforms are locally affine deformations that, when taken together, can represent arbitrarily complex deformation fields. Preliminary work has focused on modeling the texture flow of the banded segment of the seismic volume, shown in Figure 1. By tessellating a lattice of points, where each triangular patch's shape change is modeled as a single affine transform, we can track the complex motion of that region. Figure 2 shows some results.

Figure 2: beginning with a tessellation of a rectangular grid, we begin to track the complex deformation of the banded region of the seismic volume data. Although there is a dominant, global motion of the entire region, there are also more complex, local motions. These are modeled using piecewise affine transforms. By inverse warping the image back to the starting position, we can see that the texture itself changes over time. Thus, not all of the change in appearance of the texture can be attributed to it deforming.

Future Work

We are continuing the work on developing a join model of both appearance change and deformation that can be used to classify, track, and segment regions of coherent texture and motion. We are also working on learning the allowed deformations associated with particular regions as well as learning the model of appearance change that the textures undergo.

References:

[1] G. Doretto and Y. N. Wu and S. Soatto. Dynamic textures. In International Journal of Computer Vision, 2003.

[2] S. Soatto and G. Doretto. Dynamic textures. In ICCV, 2001.

[3] A. W. Fitzgibbon. Stochastic rigidity: image registration for nowhere-static scenes. In ICCV, 2001.

[4] R. Vidal and A. Ravichandran. Segmentation and Optical Flow for Multiple Moving Dynamic Textures. In CVPR, 2005.