TR#278: `Video orbits': characterizing the coordinate transformation between two images using the projective group

Steve Mann and Rosalind Picard

Many applications in computer vision benefit from accurate, robust analysis of the coordinate transformation between two frames. Whether for image mosaicing, camera motion description, video stabilization, image enhancement, aligning digital photographs for modification (e.g. ad-insertment), or their comparison during retrieval, finding both an estimate of the coordinate transformation between two images, and the error in this estimate is important.

Perhaps the most frequently used coordinate transformation is based on the 6-parameter affine model; it is simple to implement and captures camera translation, zoom, and rotation. Higher order models, such as the 8-parameter bilinear, 8-parameter pseudo-perspective, or 12-parameter `biquadratic', have also been proposed to approximately capture the two extra degrees of freedom that a camera has (pan, tilt) that are not captured by the affine model. However, none of these models exactly captures the eight parameters of camera motion. The desired parameters are those of elements in the projective group, which map the values at location x to those at location %% ${\bf x'}=({\bf Ax + b})/({\bf c}^T {\bf x} + 1)$, equation in image where the numerator contains the six affine parameters, and the denominator contains the two additional pan-tilt or ``chirp'' parameters, c.

This paper presents a new method to estimate these eight parameters from two images. The method works without feature correspondences, and without the huge computation demanded by direct nonlinear optimization algorithms. The method yields the ``exact'' eight parameters for the two no-parallax cases: 1) a rigid planar patch, with arbitrary 3D camera translation, rotation, pan, tilt, and zoom; and 2) an arbitrary 3D scene, with arbitrary camera rotation, pan, tilt, and zoom about a fixed center of projection.

We demonstrate the proposed method on real image pairs and discuss new applications for facilitating logging and browsing of video databases.

PDF . Full list of tech reports