Image banner for technology rowing detection section

"Rowing on the Boat" Detection

  In the river world, a motion-energy detector is used to compute the amount of movement on each side of the bed. This information is then used by the control program to decide if the boat is moving (i.e. passengers are "rowing") and if the people have avoided obstacles in the river by rowing vigorously on the correct side of the boat.  
Background difference image showing rotated bed Background difference image showing person moving bed
Image of rotated bed in KidsRoom from top camera Image of person pushing bed in KidsRoom from top camera

"Ready to Row!"

The algorithm must first wait until everyone is inside the boat (i.e. all on the bed). This information is determined using the tracker. Once everyone is inside, the algorithm needs to find the orientation of the bed which will be used to determine the side of the bed on which people are rowing. Even though the bed is a brightly colored (green) object, we cannot use this color information to determine its orientation due to the people sitting on top, hiding the bed color. Knowing the size of the bed from the top down tracker background-subtracted view, the algorithm waits until the number of pixels in the large blob is approximately equal to the known size of the bed. The story is used to encourage participants to be inside the bed's boundary (e.g. "Tuck your hands and feet right in, the hungry sharks are eager to sin.") When everyone is "in," the blob size is about right and the bed orientation (i.e. left and right side of bed) is computed.

Image of detected bed with ellipse drawn around it Image of person "in" the bed


Rowing on Right or Left Side

To tell if the group of people on the bed is rowing on the right or left side of the bed, the algorithm computes the difference between consecutive video frames. If someone moves quickly, a large difference between frames is detected. The magnitude of this difference is the rowing energy. Since the left and right sides of the bed are known from the initialization step, the algorithm can compute an energy measure for each side of the bed. At each time instant the energy level detected is scaled based on the maximum energy level that has been seen so far, which makes the energy values less sensitive to the number of people in the boat and minor perturbations. The control program then uses these energy measures to detect whether or not people are rowing and on which side most of the rowing is occurring. It does this by ensuring that there is at least some minimal amount of rowing and that rowing on one side is at least twice as energetic as rowing on the other side.

Image of boat energy when people rowing on left side Image of boat energy when people rowing on both sides Image of boat energy when people rowing on right side

Quicktime plug-in required

Click image.
Colored regions
are detected
left and right
rowing energy.
Left energy: .86
Right energy: .04
Left energy: .30
Right energy: .39
Left energy: 0.0
Right energy: .77

Man Overboard

The "all-on-the-bed" event detector is used to check if a person has "fallen" off the boat. If so, the room responds with a "man overboard" message and the boat stops until the person gets back onboard.

Binary background differenced image of man overboard with boat Image of man overboard


  Story - Playspace - Technology - People - Info  

The KidsRoom - Perceptual Computing Group - MIT Media Laboratory