Question #4 - Animating Action

4) What are the minimal cues for animating action? How much can motion replace realistic rendering? Does this tell us anything about the primitives of action perception or representation?

Minoru Asada

I'm not sure the meaning of this query, but the design of the primitives of action and perception representation is one of the most essential problems. "Motion Sketch" [Nakamura and Asada95] is one of our solutions.

Michael Black

In recognizing action, a significant amount of work has focused on motion, but general "appearance change" is equally important. To animate someone dancing in a long flowing dress, the changes in appearance due to the deforming texture (self shadowing, occlusion, etc) are likely to be as important as the motion information.

In work with Shanon Ju and Yaser Yacoob, we looked at the tracking of human limbs using motion techniques. A detailed examination of image sequences of people in a variety of clothing reveals that much of the "change" observed between frames is not due the semi-rigid displacement of the limbs, but rather is due to the illumination changes resulting from the non-rigid deformation of the clothing. This presents serious (possibly fundamental) challenges to approaches that base recognition solely on motion.

Another common example occurs in image sequences of people talking. The motion of the lips deforming is a significant cue, but so too is the rapid appearance/disappearance of the teeth, tongue, and mouth cavity. Yacoob, Fleet, and I have referred these non-motion changes as "iconic change".

This iconic change should not be viewed as a problem that motion estimation approaches need to work around. Rather, information about deforming clothing and appearing teeth provides important cues about the activity that is occurring. Recognition techniques should exploit both motion and other types of appearance change.

Irfan Essa

Perhaps the concepts of physics-based modeling can be employed for developing detailed representations of actions. The major benefit of physics-based methods would be that the variability of time can be easily incorporated into the analysis and allow for spatio-temporal representations. Animation and mimicking of actions is a very important by-product of this method. Again, the limitation is the domain and the context.

Yasuo Kuniyoshi

My notion of spatial and temporal attention control can be reversely exploited in the animation domain. The quality of images can be controlled so that it provides only enough details necessary for humans to grasp what's going on in the film. The problem is that we know very little about generalizing spatio-temporal attention control in action perception.

Emre Yilmaz

Aesthetically, motion capture does not easily produce the kind of wild exaggeration and dramatic motions that are so important in normal animation. However, it easily reproduces little tiny gestures, shrugs of the shoulder, subtle body language, and a sense of weight and mass-which take great talent to represent with keyframe animation or stop-motion.

Good animation-like good puppeteering, or for that matter, good mime-is rarely about trying to duplicate realistic movement. It's more often about drawing from a real movement, developing the essence of it into something larger than the reality. It's also about distilling down all the movements you could be doing to the movements that are necessary, and presenting those essentials clearly. Realistic motion feels different to watch than artistically interpreted motion. Reality is a guideline, but usually only a starting point, not the final goal.

Back to Workshop Homepage