Sensors
Actuators
Data Collection

Scenarios

Multiplayer Game
Collaborative Design
Presentation
Meeting

Features

Audio
Video
Interface State

Models

Capabilities
Graphical Models
Influence Model
Conversation Model
Game Theory
Small World Networks

 

Audio Features

Pitch/Voicing Detection Use source-filter model of vocal tract to find voiced regions Distinct signature in spectral domain Use cepstral analysis to find pitch period Group voiced components into utterances
Source Localization We want to know who is speaking when Basic principle: using timing difference to localize sound direction Match voiced components for greater accuracy Also use speaker identification features
Prosodic Features Pitch variation Speaking Rate (rate of voiced segments) Relative energy