Follow us on: GitHub YouTube Vimeo Twitter

Ubiquitous & Affective Computing

We conduct research at the intersections of ubiquitous computing, artificial intelligence and human computer interaction. With increased computing power, shrinking hardware, ubiquitous wireless networks, and widespread adoption of personal computing devices, we are entering into a new technological era of how humans interact with machines. This is made possible through embedding (at times personal and imperceptible) low-cost and low-power sensors and devices into our everyday environments.

A key focus of our group is on affective computing and wearable sensing, including:

  • Emotion recognition and behavior understanding algorithms based on machine learning techniques
  • Prototypes and infrastructure for gathering, synchronizing, and visualizing user affective states in real-world settings
  • Developing new interaction techniques including smart textiles, using physiological sensing

We study handheld and wearable sensors as a paradigm for collecting and processing affective data, across a range of domains including media consumption, automotive, fashion, immersive VR experiences, and education. Based on realistic testing grounds, collaborating with several commercial and academic partners, we have deployed our technology and infrastructure in places such as the National Theatre of China in Shanghai and the Amsterdam Dance Event in the Netherlands. Our overall objective is to create intelligent and empathic systems that both respect user privacy and appropriately react to humans and their experiences.

Topics

1. Sensing and Understanding Human Activities

  • Human activity recognition
  • Sleep monitoring
  • Sensing nightclub activity
  • Understanding urban mobility

2. Internet Of Things

  • Igor/IOTsa

3. Affective Computing

  • Multimodal emotion recognition
  • Automotive physiological sensing
  • Physiological privacy
  • Emotion-aware fashion
  • Sensing audience engagement

4. Smart Textiles

  • Interactive fashion
  • Self-actuating textiles

Funding

Videos

Open Source Infrastructure

Key Publications

  • T. Zhang, A. El Ali, C. Wang, A. Hanjalic, and P. Cesar RCEA: Real-time, Continuous Emotion Annotation for Collecting Precise Mobile Video Ground Truth Labels. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (ACM CHI), Honolulu, HA, USA, 2020.
  • A. El Ali, X. Yang, S. Ananthanarayan, T. Röggla, J. Jansen, J. Hartcher-O’Brien, K. Jansen, and P. Cesar ThermalWear: Exploring Wearable On-chest Thermal Displays to Augment Voice Messages with Affect. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (ACM CHI), Honolulu, HA, USA, 2020.
  • P. Cesar, V. Singh, R. Jain, N. Sebe, and N. Oliver New Signals in Multimedia Systems and Applications. IEEE Multimedia (IEEE MM), 25 : pp. 12-13, 2018.
  • A. El Ali, T. Stratmann, S. Park, J. Schöning, W. Heuten, and S. Boll Measuring, Understanding, and Classifying News Media Sympathy on Twitter after Crisis Events. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18), Montreal QC, Canada, 2018.
  • C. Wang and P. Cesar The Play Is a Hit - But How Can You Tell?. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition, Singapore, Singapore, pp. 336-347, 2017.
  • S. Cabrero, J. Jansen, T. Röggla, J.A. Guerra-Gomez, D.A. Shamma, and P. Cesar CWI-ADE2016 Dataset: Sensing nightclubs through 40 million BLE packets. In Proceedings of the ACM Multimedia Systems Conference (ACM MMSys 2017), Taipei, Taiwan, pp. 181-186, 2017.
  • C. Wang, J. Wong, X. Zhu, T. Röggla, J. Jansen, and P. Cesar Quantifying Audience Experience in the Wild: Heuristics for Developing and Deploying a Biosensor Infrastructure in Theaters. In Proceedings of the International Workshop on Quality of Multimedia Experience (QoMEX 2016), Lisbon, Portugal, pp. 1-6, 2016.
  • C. Wang, E. Geelhoed, P. Stenton, and P. Cesar Sensing a Live Audience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'14), Toronto, ON, Canada, pp. 1909-1912, 2014.
  • P.S. Wen Shieng, J. Jansen, and S. Pemberton Fine-grained Access Control Framework for Igor, a Unified Access Solution to The Internet of Things. Procedia Computer Science, 134 : pp. 385-392, 2018.

Theses