Recent progress in motion analysis has been achieved with systems that estimate global parameterized motion by integrating multiple constraints. The success of these approaches depends critically on the ability to segment constraints derived from different motions. Hence the problems of motion estimation and segmentation are tightly coupled. We believe it is impossible to solve these problems solely in the motion domain, and that mechanisms of spatial form analysis must be incorporated into the motion estimation procedure. We present a new framework which allows the incorporation of form information in a graceful manner. It combines concepts from perceptual organization with the powerful optimization technique of EM. We show that the algorithm is guaranteed to decrease a cost function at every iteration, and that in the absence of form information the cost function reduces to the one minimized by EM. We demonstrate that the approach can achieve good motion estimation and segmentation with challenging motion sequences.