"Rowing on the Boat" Detection
| In the river world, a motion-energy detector is used to compute the amount of movement on each side of the bed. This information is then used by the control program to decide if the boat is moving (i.e. passengers are "rowing") and if the people have avoided obstacles in the river by rowing vigorously on the correct side of the boat. |
"Ready to Row!"
The algorithm must first wait until everyone is inside the boat (i.e. all on the bed). This information is determined using the tracker. Once everyone is inside, the algorithm needs to find the orientation of the bed which will be used to determine the side of the bed on which people are rowing. Even though the bed is a brightly colored (green) object, we cannot use this color information to determine its orientation due to the people sitting on top, hiding the bed color. Knowing the size of the bed from the top down tracker background-subtracted view, the algorithm waits until the number of pixels in the large blob is approximately equal to the known size of the bed. The story is used to encourage participants to be inside the bed's boundary (e.g. "Tuck your hands and feet right in, the hungry sharks are eager to sin.") When everyone is "in," the blob size is about right and the bed orientation (i.e. left and right side of bed) is computed.
Rowing on Right or Left Side
To tell if the group of people on the bed is rowing on the right or left side of the bed, the algorithm computes the difference between consecutive video frames. If someone moves quickly, a large difference between frames is detected. The magnitude of this difference is the rowing energy. Since the left and right sides of the bed are known from the initialization step, the algorithm can compute an energy measure for each side of the bed. At each time instant the energy level detected is scaled based on the maximum energy level that has been seen so far, which makes the energy values less sensitive to the number of people in the boat and minor perturbations. The control program then uses these energy measures to detect whether or not people are rowing and on which side most of the rowing is occurring. It does this by ensuring that there is at least some minimal amount of rowing and that rowing on one side is at least twice as energetic as rowing on the other side.
Click image.
Colored regions
are detected
left and right
rowing energy.Left energy: .86
Right energy: .04Left energy: .30
Right energy: .39Left energy: 0.0
Right energy: .77Man Overboard
The "all-on-the-bed" event detector is used to check if a person has "fallen" off the boat. If so, the room responds with a "man overboard" message and the boat stops until the person gets back onboard.
