Phones know how you feel

Wednesday 29 September 2010

New phone technology knows if you’re sad or happy.

When you are in a meeting at work, are you happy or sad? Or is being in the pub with a small group of friends likely to make you more emotional?

New research developed by the University of St Andrews can explore the effects of location and the company you keep on your mood, all via mobile phone technology.

The new system, created by a researcher at St Andrews in collaboration with Cambridge University, could help psychologists better understand human emotional behaviour.

Called “EmotionSense”, the system that uses sensors and speech-recognition software, has been successfully road-tested by researchers. The findings will be reported today (Wednesday, 29 September) at the Association for Computing Machinery’s conference on Ubiquitous Computing in Copenhagen.

The development means that in-built phone sensors in standard smart phones will be able assess how people’s emotions are influenced by factors such as their surroundings, the time of day, or their relationships with others.

St Andrews researcher, Dr Mirco Musolesi explained, “EmotionSense is potentially a revolutionary platform for social psychologists for understanding human behaviour in an unobtrusive way.

“The possible applications are many: from finding answers to long-standing social psychology problems, such as the influence of interactions on emotional states of people, to supporting behavioural interventions for preventing depressive conditions or promoting well-being in general.”

Early results suggest that the technology could provide psychologists with a much deeper insight into how our emotional peaks – such as periods of happiness, anger or stress – are related to where we are, what we are doing or who we are with when near a mobile phone.

Dr Musolesi explained, “We are not only inferring emotions during phone calls, but the emotions of the people close to the phones whether in a pub or during a meeting at work. The mic of the phone is used as a sort of an open mic in a room that detects voices of all the speakers, identify them and infer their emotions.”

EmotionSense matches the user’s voice patterns to recorded speech samples that represent 14 different emotional categories, which are grouped into five broader categories – “Happy” (such as elation, or interest); “Sadness”; “Fear”, “Anger” (which includes related emotions such as disgust) and “Neutral” (such as boredom or passivity).

The data can then be cross-reference with other information picked up by the phone, such as where you are -via GPS – and who you are with and when – via bluetooth technology.

The researchers say that since the analysis is made on the phone itself, the data can be immediately discarded after use, thus protecting user privacy.

During trials, subjects were asked to keep a diary recording their emotional state, and when compared with the EmotionSense results, 70% of cases were in agreement with how they actually felt.

Interestingly, a user’s location appeared to have a pronounced effect on their state of mind. “Happy” emotions dominated the data when they were in residential locations, while in the workplace, “sad” emotions became the norm.

The researchers also found that users exhibited more intense emotions in the evening than in the morning and that people tended to express their emotions far more in smaller groups than in larger crowds.

Dr Musolesi, a computer scientist at St Andrews, was responsible for the design and implementation of power-efficient algorithms that can be built into existing sensors (such as those used in microphones) to detect human emotions and interactions.

He commented, “The key novelty is that the software can be deployed on users’ phones without requiring any additional hardware. For this reason, the platform is also extremely cost effective for large-scale studies and interventions.”




Issued by the Press Office, University of St Andrews

Contact Gayle Cook, Senior Communications Manager
on 01334 467227 / 462529, mobile 07900 050103,
or email [email protected]
Ref: Phone feelings
View the latest University press releases at

Category Research

Related topics

Share this story