The HyperPlex system assembles work from three different research groups: Vision and Modeling, Interactive Cinema, and Autonomous Agents.
Research in the Vision and Modeling group at the Media Laboratory allows interaction without the use of cumbersome device or cables. When users are in front of a big display screen, the use of a mouse and a keyboard to issue commands to the system is extremely limiting. It is much better if interaction takes place by the use of gesture and voice recognition systems (multi-modal interaction). The user becomes a 3D-mouse that can point at or highlight different portions of the screen, and give simple commands [Darrell et al. 1994].
Research at the Interactive Cinema group has created a video browsing environment for large databases of digital video [Davenport1993], by making the spatial dimension of the displayed information (i.e., where things appear on the screen) as important as its temporal dimension (i.e., what comes after what in a video sequence). The conceptual characteristics of each video clip are associated with physical locations, and users browse the database by steering through this ``conceptual space.'' The challenge that this research group has been faced with is the construction of storyteller systems that handle variable, non-linear, multi-threaded narratives.
Research by the Autonomous Agents group has considered how computer programs (``agents'') should respond to user's activity given their internal motivation, past history and a perceived environment with its attendant opportunities, challenges and changes [Maes1995]. Moreover, the pattern and rhythm of the chosen activities should be such that it neither dithers between multiple activities nor persists too long in a single activity. It should be capable of interrupting a given activity if a more pressing need or if an unforeseen opportunity arises.
Finally, navigation in such a graphical virtual environment implies considering not only the construction of a visually compelling virtual world but also providing the users with visual cues that help them orient themselves in the simulated reality.