Ideally, an affective wearable would be able to sense and recognize patterns corresponding to underlying affective states, and respond intelligently based upon what it has sensed. We know of no computers that can do all of this yet; there are many difficult problems which need to be solved first. However, we have a prototype that achieves part of these goals, which we will now describe. The rest of this paper will focus on the results we have achieved so far in our efforts to develop the sensing and annotation aspects of an affective wearable computer. (The pattern recognition aspects are also important; these will be presented in a later publication.)
The current version we have built of an affective wearable is an augmentation of Thad Starner's design 1 [Sta95] using the PC 104 board standard and the Private Eye display, shown in Figure 1. Attached to this is a medically approved bio-monitoring system made by Thought Technologies, which has the ability to simultaneously monitor respiration, skin conductivity (GSR), temperature, blood volume pressure (BVP), heart rate (from BVP), and EMG (electromyogram, for muscular electrical activity). All of these can be sensed painlessly from the surface of the skin. Future versions of the system may include audio and video inputs and displays, wireless links to the Internet and wireless localized sensors.
Current functionality includes the monitoring of four sensors by a linux based operating system. The input from the four sensors can be displayed on a text-based screen such as the Private Eye with an option for concurrent user annotation. The annotations are automatically time-stamped by the system and stored in a separate log file. In the near future, we hope to add a third log file recording the user's location at periodic intervals using GPS for outdoors, and a system of fixed infrared location broadcasting stations for in<side our lab.
The four biometric sensors can be sampled at up to 20 samples per second by the linux based ProComp system which allows an hour of data to be stored uncompressed in 1.125 MB as four byte floating point numbers. We are currently addressing the problem of down loading accurately time stamped data from the wearable via PCM (Sierra wireless modem) so that monitoring and recording of data can occur continuously even at higher sampling rates.
A difficult challenge of affective computing research is to determine which features of the sensor information should be considered salient, both to reduce the amount of data that is stored and transmitted, and to improve the analysis of the data. For example, it could be that a user is interested in monitoring her relative stress level over an extended period of time. Salient features for measuring stress could include the slope of the skin conductivity, average heart rate, average respiration rate or a combination of these and other signals. This data could easily be assessed and stored at a much lower saving rate (e.g., once per minute). At the end of a day or week, the user could view her daily stress profile. With intelligent annotation from the user--comments such as ``begin work,'' ``end work,'' ``begin lunch,'' ``end lunch,'' ``begin meeting supervisor'', ``begin driving''--the stress profile could then be sorted by activity and presented to the user in a format which communicates the relative levels of stress.