Wearable device data shows promise in predicting future emotional and mental states

As low-cost mobile and wearable sensors find increased uses, research is emerging that uses them to track and analyse mental well-being, productivity and behaviours. The lack of open datasets collected in real-world contexts is, however, limiting potential research advances in affective computing and human-computer interaction that rely on such data.

Researchers at Khalifa University have collaborated with colleagues in South Korea to investigate the potential of wearable sensor devices in gathering data that correlates with mental and emotional states. Their study, conducted on a sample of 77 students over seven days, has been compiled into a dataset called K-EmoPhone.

The three-year collaboration between Khalifa University and the Korea Advanced Institute of Science and Technology (KAIST) focused on the regular collection of data aimed at revealing aspects of the participants’ mental state, including their emotions, stress, and attention levels.

“The connections forged between the Khalifa and KAIST teams will hopefully lead us to further collaborative research to advance potential for our methods to be incorporated into commercial smart devices, such as smart watches and other wearable technology,” says Leontios Hadjileontiadis of Khalifa University’s Department of Biomedical Engineering.

The study harnessed data from smartphones and commercial consumer wearable devices, which captured environmental and physiological signals, as well as movement patterns. Measurements included parameters such as body temperature, heart rate, skin electrical resistance, light exposure, and caloric expenditure linked to activity and body maintenance.

This direct physiological monitoring was complemented by data from the participants’ responses to a smartphone app questioning them about their feelings and activities. The assessments occurred at random intervals up to 16 times each day, allowing the primary sensor data to be contextualised within participants’ emotions and activities at different times of the day.

The data was then combined with that from pre and post-study surveys of the participants’ mental and emotional state, especially their ongoing sense of well-being, stress, or depression.

Taken together, the results confirm that data from wearable devices can indeed be used to assess mental and emotional well-being and can also be used to predict how these issues may be developing in an individual’s life. The findings advance the understanding of the ability of wearable technology to monitor more subtle aspects of an individual’s condition.

“I expect that our work will help develop new opportunities for healthcare professionals and researchers to analyse mental and cognitive states with smartphone and wearable sensor data,” says systems engineer Woohyeok Choi of the KAIST team.

The K-EmoPhone procedure and dataset is at a relatively early proof-of-concept stage, and the researchers acknowledge the need to refine the software for broader applicability. They are, however, confident of wider general and commercial opportunities.

“Future enhancements could ensure compatibility with different commercial software systems and seamless integration into existing and emerging wearables such as smartwatches and other devices,” says Hadjileontiadis.

Reference:

1. Kang, S. et al. K-EmoPhone: A Mobile and Wearable Dataset with In-Situ Emotion, Stress, and Attention Labels. Scientific Data 10, Article number 351 (2023). | Article