22 McGrath Highway, Suite 6

Somerville, MA 02143

Morgan Hall 197

Boston MA 02163

ymoon@hbs.edu

MIT Media Laboratory

Room E15-383, 20 Ames Street, Cambridge, MA 02139

picard@media.mit.edu

A human-computer interaction (HCI) agent was designed and built to support users in their ability to recover from negative emotional states, particularly frustration. The agent uses social-affective feedback strategies delivered to the user with text-only interaction. The agent’s effectiveness was evaluated against two control conditions: (1) user’s emotions were ignored, and (2) users were able to report problems and "vent" their feelings and thoughts to the computer. Behavioral results showed that the agent was significantly more effective than the control conditions in helping relieve frustration levels.

User emotion, affective computing, social interface, frustration, human-centered design, empathetic interface

Use of computer technology often has unpleasant side effects, some of which are strong, negative emotional states. These negative emotional states can affect not only the interaction with the computer, but productivity, learning, creativity, and overall well-being [1, 2]. The HCI community knows this problem well; indeed, preemptively reducing or eliminating user frustration has long been a goal of this community. Nonetheless, user frustration stemming from interaction with computers seems here to stay. This begs the question: We know what designers can do to try to minimize user frustration, but once a user becomes frustrated, what can and should be done to reduce these strong negative feelings? Specifically, what role can the computer play in helping a frustrated user? This paper suggests a new approach, and provides evidence of its effectiveness.

The basic idea is to give a computer certain social-affective skills that humans use in helping one another alleviate frustration. These skills include a technique known as "active listening" combined with careful emulation of empathy and sympathy. The motivation and details of this approach can be found in [3], including citations supporting the effectiveness of the three components of this approach in human-human interaction. Since recent work in HCI suggests that humans may respond to computers socially [4], the approach of trying to imitate what skilled humans do seems a promising one for dealing with user frustration.

The study consisted of a 2 x 3, between-subjects, full-factorial experiment (see Table 1 below).  

Subjects (N=70) were tested individually. There were 11 or 12 subjects in each condition, with approximately equal numbers of males and females in each. Subjects were told they had a chance to win $100 for testing a new web-based computer game. Upon arrival to the lab, they were seated at a computer displaying two windows: a Netscape browser with a novel graphical adventure game interface and a text window with directions for the game.

Each subject played the game for 5 minutes. During the game, half of the subjects were exposed to simulated web-delays, in which the character froze on screen but the on-screen timer continued to advance (DELAY condition). The other half experienced no delays (NO-DELAY condition). These two conditions corresponded to high-frustration and low-frustration, respectively. (A full-factorial ANOVA confirmed that participants in the DELAY condition rated their frustration level significantly higher after this first game than participants in the NO-DELAY condition, F(1, 64) = 4.54, p<.05.)

After 5 minutes, the game stopped automatically, and the computer prompted the subject to evaluate the game by answering a series of questions. Depending on condition (IGNORE, VENT, or AFFECT-SUPPORT), subjects received one of three online questionnaires. The questionnaires were designed so that time spent filling them out did not differ significantly across conditions. In the IGNORE condition, subjects were asked closed-ended questions that did not involve emotions or provide any opportunity to report a problem like web delays. In the VENT condition, subjects were asked open-ended questions that gave them the opportunity to report the relevant problem, as well as their emotional state. In the AFFECT-SUPPORT condition, subjects were asked mostly the same questions as in the VENT condition; however, after the computer asked how frustrated the user was feeling, the computer gave feedback based on the user’s reported frustration level. Feedback included active listening, e.g., "Wow, it sounds like you felt really frustrated playing this game."; means for repair and correction, e.g., "Is that about right?" (user choices displayed); a statement of empathy gauged to the reported frustration level, e.g., "That must feel lousy. It is no fun trying to play a simple game, only to have the whole experience derailed by something out of your control;" and a sympathy statement, e.g., "This computer apologizes to you for its part in giving you a crummy experience." Friendly interactive language was deliberately used across all three questionnaires so that the affect-support agent would not appear to be friendlier than the two controls.

After the questionnaire, all subjects were asked to play a non-delay version of the same game for at least 3 minutes. The quit button would appear at that point, and they could play longer if they wished, up to 20 minutes, but the game was designed to be boring and subjects were given no incentive to play longer than 3 minutes. Behavior was measured as how long each subject chose to continue to interact with the system playing the game.

The key prediction was that DELAY subjects, who were experiencing high levels of frustration resulting from the delays in Game 1, would feel more emotional relief after experiencing the AFFECT-SUPPORT condition, in comparison to subjects in the other two conditions. The AFFECT-SUPPORT subjects were expected to feel more positively toward the source of their frustration—the game and networked computer system—and were therefore predicted to play longer in Game 2 than subjects in the DELAY/IGNORE and DELAY/VENT conditions.

The results supported this prediction. A full-factorial ANOVA revealed two main effects. First, there was a main effect for feedback-type, F(2, 64) = 8.00, p<.01. Planned orthogonal comparisons (controlling for inflated significance levels associated with multiple comparisons) indicated that subjects in the AFFECT-SUPPORT condition played Game 2 significantly longer than participants in either the IGNORE condition (t(45) = 2.63, p<.01), or the VENT condition (t(44) = 3.97, p<.01). There was no significant difference in the game-playing behavior between participants in the IGNORE and VENT conditions. There was also a main effect for frustration, such that participants in the DELAY condition played Game 2 for a significantly longer time than participants in the NO-DELAY condition, F(1, 64) = 9.20, p<.001. The interaction effect was not significant. No significant results were found when gender, emotional arousability, and prior game play experience were tested on the main behavioral results with 3-way ANOVAs.

Several explanations have been considered for the main finding that the AFFECT-SUPPORT condition led to longer play the second time around [3]. Many people play games to relax, and subjects in the DELAY condition all played longer than those in the NO-DELAY condition. However, this difference can be attributed to the rebound effect: subjects who experienced delays in the first game did not experience them in the second game—a pleasant surprise which led to their playing longer. The rebound effect makes it meaningless to compare times across the DELAY/NO-DELAY conditions. What is significant is that within the DELAY condition, those who experienced the affect-support played significantly longer than those who did not. The same result held within the low-frustration NO-DELAY condition. We also considered alternate explanations, such as that subjects might have played longer if the affect-support aggravated them. Careful analysis of the data suggests that this explanation (and many others; see [3]) are not supported. Subjects did not report increased irritation after interacting with the agent. The best explanation to date is that a text-only interaction, which carefully applies social-affective skills known to work in human-human interaction, can be used by a computer to provide relief of negative emotional states related to frustration, as manifest in subsequent user behavior toward the object of the negative emotion.

These results suggest that designers should consider the user’s emotional state as an interactive factor in the design process. Not only should designers aim to eliminate sources of frustration up front, but they should also consider building behaviors into the system to address emergent, as well as ongoing, user frustration.

Important next steps include giving the system the ability to recognize when users are frustrated, and exploring variations on the affect-support system’s design. Many questions remain, e.g., are both empathy and sympathy required to maintain effectiveness? But the challenge has begun: how can computers best respond to the emotions of their users?