TR#221: Layered Representation for Motion Analysis

John Y. A. Wang and Edward H. Adelson

Article available in:
Proc. of the IEEE Comp. Vis. and Pat. Rec. (CVPR)
pp. 361-366, New York, June 1993.

Similar version appears in:
J. Y. A. Wang and E.H. Adelson, "Layered Representation for Image Sequence Coding",
Proc. of the IEEE Int'l Conf. on Acoustics, Speech, and Signal Processing,
vol. 5, p. 221-224, Minneapolis, Minnesota, April 1993.

Standard approaches to motion analysis assume that the optic flow is smooth; such techniques have trouble dealing with occlusion boundaries. The most popular solution is to allow discontinuities in the flow field, imposing the smoothness constraint in a piecewise fashion. But there is a sense in which the discontinuities in flow are artifactual, resulting from the attempt to capture the motion of multiple overlapping objects in a single flow field. Instead we can decompose the image sequence into a set of overlapping layers, where each layer's motion is described by a smooth flow field. The discontinuities in the description are then attributed to object opacities rather than to the flow itself, mirroring the structure of the scene. We have devised a set of techniques for segmenting images into coherently moving regions using affine motion analysis and clustering techniques. We are able to decompose an image into a set of layers along with information about occlusion and depth ordering. We have applied the techniques to the ``flower garden'' sequence. We can analyze the scene into four layers, and then represent the entire 30-frame sequence with a single image of each layer, along with associated motion parameters.