Teaching Emotion Recognition with Interactive Media & Wireless Expressive Toys
CHAPTER ONE: INTRODUCTION Chapter Index
- An Observation
- affect scenario:
- Imagine you’re waiting to meet an old friend you haven’t seen in years. You’ve been looking forward to this opportunity for some time and you have arrived ten minutes early in anticipation of this exciting moment. After fifteen minutes you feel quite anxious. You wonder if you have the right meeting place and check your notes to be certain that this is the correct park, the Japanese Gardens, and then you re-check the time. For the next five minutes you wait patiently, but after ten more minutes pass you imagine something may have happened. Then, ranges of feelings go through you: concern, disappointment, frustration, and then anger. After ten more minutes, your friend finally arrives. You look closely at her to see if you can catch a hint of what may have caused the delay. You read distress and angst on her face and the angry feelings you felt subside as you try to support her and listen empathetically to what happened that caused her to be late. This situation has affected your initial embrace.
- autistic scenario:
- While you’re conversing you appreciate that your friend chose the Japanese Gardens as a meeting place because of its beautiful ambiance. Sitting on a bench overlooking the pond, you both admire the scenery. Suddenly a little boy sits between you and your friend. He is intensely engaged in watching the sun’s reflection wiggling off the top of the water and the brilliantly colored Koi swimming in the pond. You and your friend both look at each other in surprise. Your friend attempts to get this young boy’s attention. She asks him what his name is in a happy lilting voice, yet the boy doesn’t respond. She asks again as she bends down to look at him. The boy says, "goldfish big," but doesn’t look at her. You begin to feel a little awkward. You hear a mother calling a boy’s name. You look toward the voice and see that she recognizes her son because she is quickly walking towards you with a concerned expression on her face. This little fellow seems to be hypnotized by the fish and shows no reaction to his mother’s voice. When the mother arrives, she gently reaches for her son by firmly grabbing his wrist and preventing him from squirming away from her, then apologizes to the women for his behavior. She explains that he has autism. After they leave, you realize the little boy seemed unaffected by what just happened.
Explanation
Motivation
The goal was to build a computer system that would organize and display emotionally expressive media and guide the child’s interaction with this media in an effort to facilitate learning of emotion expression recognition. The system is an example of Affective Computing, "computing that relates to, arises from, or deliberately influences emotions" (Picard 97, p 3). A new form of interaction was also explored in place of the traditional keyboard and mouse – the use of dwarf-dolls as a physical interaction. Through these objectives, the research explores the potential for using computing and physical interfaces in therapy.
The thesis question asks: "Through the use of engaging multimedia elements and affective computing, can autistic children be engaged in their interaction with the application long enough to possibly learn to recognize basic emotions?" If successful, could this potentially help them with social and affective communication in real life situations? Generalization to everyday life situations will not be answered in this thesis, although the attempt is to identify the merits of using technology to facilitate whether autistic children can engage in the use of an automated system to learn to recognize different displays of emotional expression.
It is our goal to create a computer-based affective system that can synthesize social communication in order to promote the recognition of affective information. The system design builds on empirical findings regarding what engages autistic children (inanimate objects) and how they process information (visual images.) The computer application offers a puzzle matching game with video clips to match emotional expressions. This multimedia tool presents different ways that an emotion may be communicated. Analogous to a foreign language, autistic children may learn social communication as a second language and choose to use it in social situations.
Additionally, through an automated approach to teaching emotion recognition, this type of application may be helpful in other teaching methods. The most common treatment for young children with learning disabilities is specialized instruction prior to mainstream primary education in an attempt to prepare them for typical assimilation by the time they are six years old and start school. Pre-schooling is commonly referred to as early intervention and behavior analysis is one type of intervention. This approach to teaching behavior and social skills may include discrete trial sessions; time spent one-on-one with a practitioner to learn a specific task. Technically, discrete trial is a structured time trial where a particular stimulus prompts a specific response and receives a consequence that will either reinforce that response or will promote a different response. The time demand on the practitioner using discrete trial or other behavior-analytic treatments might be long and highly expensive due to one-to- one teacher to student ratios, manual data collection, data analysis, and procedural modifications. An automated intervention application with novel play-like elements might offer a complimentary method for behavior analytic training procedures. A goal for the application presented in this thesis is emotion recognition, though its success suggests the possible benefits for other areas of development.
Affective Social Quest
Recognizing and expressing affect is a vital part of social participation.
Unfortunately, those with autism have their primary learning disability
in this area with other deficits in language, motor and perceptual development.
Their development of social communication is very low compared to neurologically
typical children who learn social cues naturally while growing up. Along
the autistic spectrum, varying deficits distinguish the child’s level of
autism as compared to a normal child who is considered to have no deficits.
At one end -- where the child has few deficits -- is the high functioning
end of the spectrum. Asperger children, on the other hand, are high functioning
children who are distinguished by their typical verbal ability, and sometimes
are not labeled with autism but referred to as those with Asperger syndrome.
High-functioning or Asperger children who are especially bright with minimal
language deficits have a self-awareness that socially they are supposed
to blend in. At the same time they are aware that they don’t blend in and
that they don’t know how to (O’Neil 99). In trying to comprehend social
nuances in communication or social behavior to blend in during everyday
interaction, they get frustrated, not only with themselves but with their
teachers too, and often give up learning. What may help an autistic child
in this case is an ever-patient teacher.
This thesis presents an approach to creating that teacher: a persistent and unresentful aid that progressively introduces basic emotional expressions, guides recognition development through matching, and records the child’s success. The system embeds multiple subsystems to teach emotion recognition to autistic children with a heterogeneous disorder. Children with this type of disorder have a wide variety of IQ’s – and a wide variety of developmental levels – and need various systems, or ways, to interact with an environment appropriate to their deficit level. Although the application developed for this research does not come close to the abilities of a highly trained human practitioner, it is designed to offload some of the more tedious parts of the work.
With a computer running video that communicates with independent toy like objects, the system synthesizes interactive social situations in order to promote the recognition of affective information, as shown in figure 1. This system will not tire because of impatience and can be a safe place for the child to explore. We call it ASQ, for affective social quest. The goal of ASQ is to provide an engaging environment to help children -- specifically autistic children -- visually recognize social displays of affect.
Chapter One Index
Chapter Title:
Introduction
Explanation
Motivation