EmoteMail

• Authors: Jussi Ängeslevä;Carson Reynolds;Sile O'Modhrain
• Venue: SIGGRAPH '04 ACM SIGGRAPH 2004 Posters
• Year: 2004

Email has become a central communication channel for private and professional exchange. Its format remains equally neutral regardless of the relation to the recipient. While writing remains an excellent vehicle to communicate tone and emotion, this can sometimes be a painstaking and tedious process, and requires considerable skill.EmoteMail is an email client that is augmented to convey aspects of the writing context to the recipient. The client captures facial expressions and typing speed and introduces them as design elements. These contextual cues provide extra information that can help the recipient decode the tone of the mail. Moreover, the contextual information is gathered and automatically embedded as the sender composes the email, allowing an additional channel of expression.Hugo Lius EmpathyBuddy [Liu et al. 2002] and the chillies in email client Eudora also attempt to give email contextual tone by analysing the textual content of the message. While these approaches succeed in affectively annotating the email message without any extra effort, we feel that they fail to consider the personality of the sender and have no comprehension of the writing context. For example, sarcasm could be misinterpreted by both of these approaches as negative affect.In contrast, our approach relies on the relationship between the recipient and sender. The recipient can examine the senders facial expressions as well as the paragraphs the sender spent the most time writing. We think that this extra information helps convey tone especially when the recipient is familiar with the senders non- verbal expressions.We use subtle graphic design elements to relate facial expression and typing speed, leaving the textual message as the main focus. We adopt Edward Tuftes approach in information design, suggesting that when reading data-rich content, one needs prior understanding of its language. In this case, we rely on commonly shared understanding of non-verbal language. One drawback of making the contextual features more subtle is that they will increasingly assume greater familiarity between communicators. It is interesting to consider how the choice of subtle cues compares to more attention grabbing techniques.Dynamic typography is one technique that is also used for attaching emotions and personality to written language. Text that scales, moves, changes colour, typeface and proportions, has been successfully demonstrated in a poetic context [Forlizzi et al. 2003; Bodine and Pignol 2003]. We think the drawback of this approach is that the written content of the text recedes to the background and becomes more difficult to access. We feel that in email the text should be at the forefront, with the contextual cues playing a supporting role.Instead of dynamically modifying individual characters, EmoteMail relates contextual cues to each paragraph. The prototype uses a camera and a timer as sensor inputs to capture additional information relating to the paragraph. Reminiscent of the commonly used smileys, the EmoteMail client annotates every paragraph with a small black and white thresholded image of the face of the writer. Each paragraph also includes a background colour representing how much time the paragraph took to compose.If the sender changes their facial expression (perhaps as part of an emotionally meaningful communication) then the small camera grab beside each paragraph reflects this. By capturing a snapshot of the face of the writer with every paragraph, the system attempts to display the fluctuation of the emotions throughout the message, rather than attempting to summarize the whole message as a certain mood.Likewise, the time spent on writing a paragraph is measured in relation to all the other paragraphs written. Typing speed may denote more thought having been put into crafting a paragraph. The typing speed can show what parts of the message have been copied and pasted and it might also suggest the attention level of the person writing the message. Ultimately, the interpretation is left to the recipient.Informal testing of EmoteMail has been informative. While there are some issues to be resolved, the initial response has been largely positive. The ease of usage and the automatic capturing of the context is appreciated. However, the clients live video preview depicting the senders face can be distracting. This raises some good design questions: should senders be given the possibility to edit the contextual features? Or should senders see only the plain text? We are also considering a large range of other contextual inputs for future versions, ranging from pressure and skin conductivity sensors to audio analysis and reading of the state of the computer at the time of writing. We are also considering rewriting EmoteMail as a plug-in to a popular email client to facilitate more real life user studies. The primary benefit of EmoteMail is that it will allow us to conduct further research to understand if contextual information in email improves the quality of communication in a measurable way.