Follow us on: GitHub YouTube Vimeo Twitter

Master's Thesis Topics 2021

Publication date: 2020-09-23

The Distributed & Interactive Systems group at CWI has new open positions for motivated students who would like to work on their Master’s thesis as an internship in the group. Topics include smart textiles, activity recognition, physiological sensing, virtual reality, point clouds, Internet of things and web technologies. Keep reading for more information about research topics, requirements and contact information.

One-shot Interaction for Emotion Recognition using Physiological Signals

Contact: Tianyi Zhang (tianyi@cwi.nl), Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Traditional approaches of emotion recognition rely either on custom models trained in situ, or general models pre-trained on existing datasets. The first kind of methods need users to annotate their emotions in situ. Since the data is collected in-situ as well, accuracy tends to be quite high while the burden of annotation to the user is also high. The second kind of methods build the recognition system based on the information learned from other datasets, which leads to lower annotation burden to the user. However, since there is no data and annotation specifically for one individual user, the recognition accuracy on that user tends to be less accurate. To overcome these challenges, this work will develop an emotion recognition system aiming to provide high classification accuracy, while minimizing the annotation burden of users. The system will use self-supervised and one-shot learning to first automatically learn the data representation in situ and only require users to input their emotions (annotation) when labels are needed to fine-tune the algorithm for better accuracy. The self-supervised learning algorithm will automatically determine the annotation frequency to make a tradeoff between annotation burden and recognition accuracy. The work of this project includes hardware and software implementation of emotion recognition using self-supervised learning techniques. The work also includes experiments to evaluate the user burden of annotation and the recognition accuracy compared with other methods (i.e., systems trained in situ and pre-trained on other datasets).

Skills:

  • Required:
    • Knowledge about machine learning, especially about one shot learning and self-supervised learning
    • Knowledge about time series signal processing
    • Good programming skills in python, Java (for Android)
  • Recommended:
    • Deep learning
    • Human computer interaction

Literature:

  • CHI 2020: “Automated Class Discovery and One-Shot Interactions for Acoustic Activity Recognition” https://dl.acm.org/doi/10.1145/3313831.337687
  • Sarkar P, Etemad A. Self-supervised learning for ecg-based emotion recognition. InICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2020 May 4 (pp. 3217-3221). IEEE.
  • Banville H, Moffat G, Albuquerque I, Engemann DA, Hyvärinen A, Gramfort A. Self-supervised representation learning from electroencephalography signals. In2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP) 2019 Oct 13 (pp. 1-6). IEEE.
  • Li Fei-Fei, Rob Fergus, and Pietro Perona. 2006. Oneshot learning of object categories. IEEE transactions on pattern analysis and machine intelligence. IEEE, 594- 611.
  • Oriol Vinyals et al. 2016. Matching networks for one shot learning. Advances in neural information processing systems (NIPS). 3630-3638
  • Saeed A, Ozcelebi T, Lukkien J. Multi-task self-supervised learning for human activity detection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2019 Jun 21;3(2):1-30.

Posed vs. Spontaneous Smile Dataset using Face, Muscle, and Physiological Sensing

Contact: Gerard Pons gerard@cwi.nl, Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Facial expressions play an important role in daily social life and communication between people. It is a non-verbal way for people to show their emotions and intentions. A special type of facial expressions are micro-expressions. Micro-expressions occur when people try to hide their true feelings and emotions. This can be deliberate concealment (suppression) or unconscious concealment (repression). The recognition of micro-expressions is useful for different applications such as security, interrogations and clinical diagnosis. It is hard for people to recognize micro-expressions because of the short duration and low intensity. Therefore in the last years research about the automatic recognition of micro-expressions is getting more attention. Most of these works use RGB videos and images for automatic recognition. This project asks: How to collect a spontaneous micro-smile dataset that contains thermal videos, RGB videos and EMG signals? How well does micro-smile detection from thermal videos compare with other modalities (e.g.RGB, EMG, EDA, HR)? The main objective of this work is creating a new spontaneous micro-smile dataset with synchronised thermal videos, RGB videos and EMG signals, as well as baseline performance measures. This is a joint project with Monica Perusquia-Hernandez at NTT (Japan).

Skills:

  • Required:
    • Machine learning
    • Computer vision
  • Recommended:
    • Time series signal processing
    • Human computer interaction

Literature:

Developing and Comparing Novel Emotion Elicitation Methods

Contact: Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Affect is a fundamental aspect of internal and external human behavior and processes. While much research has been done on eliciting emotions, it remains a challenge what is the most effective method(s) for inducing emotions, and under which context. In this project, you will design and build novel multi sensory elicitation techniques: these can be visual, auditory, haptic, smell, taste, or heat. You can focus on one or more, with the goal of providing an unobtrusive and enjoyable experience that can elicit different emotion states.

Skills:

  • Required:
    • Hardware prototyping (e.g. Arduino)
    • Experiment design (controlled, field)
    • Statistics
    • Interest in human computer interaction

Literature:

Emotion Annotation across Smell, Taste, and/or Temperature

Contact: Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Our annotations are discrete, but our sensations are continuous. In this project, we aim to investigate multi-sensory emotion annotation techniques by actuating our senses. In this project, you will focus on one of these sensations, or perhaps a combination, and investigate different techniques of describing felt sensations. You will need to be comfortable building multi-sensory actuators, whether haptic, olfactory, or taste-based interfaces.

Skills:

  • Required:
    • Electronics & hardware prototyping (e.g. Arduino)
    • GUI development (mobile or desktop)
    • Experiment design (controlled)
    • Controlled user studies
    • Statistics

Literature:

Design and Development of a Steering Wheel for Emotion Annotation

Contact: Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl), Kaspar Jansen (Design Engineering - TU Delft)

Emotion recognition has moved away from the desktop, and on to the road, whether in automated or non-automated vehicles. This requires collecting precise ground truth labels in such settings, that do not pose driver distraction (whether the primary task is driving or situation monitoring in the case of automated driving). This project asks: How can drivers enter their mood while driving? This is a joint project with the Design Engineering department at TU Delft. It will require prototyping emotion input techniques on the steering wheel, and evaluating them in a desktop-based driving simulator study to ensure high usability of the wheel concept and high quality of the collected annotations.

Skills:

  • Required:
    • Hardware prototyping (e.g. Arduino)
    • Sensors and input techniques
    • Design engineering methods
  • Recommended:
    • Controlled user studies
    • Statistics

Literature:

Exploring Emotion-reactive Wearables using Physiological Sensing

Contact: Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Wearable biotech fashion is becoming a recent trend, however we still know very little on what the best means of visualizing such biometric data. This is a project to explore the intersection between fashion, aesthetics, and wearable biotech sensors. The project should result in a series of emotion-reactive wearable prototypes, and ideally evaluated in the field. This can be a joint project with the Design Engineering department at TU Delft.

Skills:

  • Required:
    • Hardware prototyping (e.g. Arduino)
    • Human computer interaction
    • Textile fabrication methods
    • Multimodal output

Literature:

ThermalWear II: Exploring Wearable Thermal Displays to Augment Human-Machine Interactions with Affect

Contact: Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Thermal stimulation is an intrinsic aspect of sensory and perceptual experience, and is tied with several experience facets, including cognitive, emotional, and social phenomena. The capability of thermal stimuli to evoke emotions has been demonstrated in isolation, or to augment media. This project will build on our prior work (see CHI 2020 paper “ThermalWear” below), and extended our ThermalWear prototype in one of several ways: (a) developing a user calibration model (b) multi-site actuation (c) dialogue systems (d) multimodal feedback. The project should result in tangible prototypes, and evaluated in a controlled study or in the field. This can be a joint project with the Design Engineering department at TU Delft.

Skills:

  • Required:
    • Hardware prototyping (e.g. Arduino)
    • Human computer interaction
    • Fabrication
    • Thermal and/or multimodal output

Literature:

FaceEmotionInput: Using Facial Expressions for Mood Input on Smartphones

Contact: Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Tracking and recognizing emotions has moved away from the desktop, and on to mobile and ubiquitous settings. We now can enter how we feel on mobile apps, and even on smartwatches. However, entering our moods is cumbersome. In this project, you will devise an unobtrusive, low(or no)-burden self-report collection strategy. The idea is to have a passive sensing method, that does not require active annotation from the participant. You will build on our own prior work (e.g., Face2Emoji, RCEA), and develop a facial expression-based input method. This can be evaluated remotely, or in controlled, user studies.

Skills:

  • Required:
    • Android development
    • Human computer interaction
    • (Basic) machine learning and computer vision
    • Quantitative research methods

Literature:

CakeVR: Designing and Evaluating Interaction Techniques for Making Cakes in Virtual Reality (VR)

Contact: Jie Li (j.li@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

We have developed a social VR tool for pastry chefs and clients to remotely co-design cakes (Mei, 2020), including using natural gestures to manipulate the cake size, decorating the cakes with pre-designed cake components (e.g., cream flowers, fruits), and showing an instant 3D visualization of the co-design outcome. However, many interaction techniques in the current prototype need to be improved. For example, how to precisely position a decoration on the virtual cake, how to perform mid-air sketch stably, how to calculate the dimension of the cake, and how to duplicate and group decorations.

In this project, you will identify the necessary interactions for collaborative creative tasks such as making cakes, and design and implement the interaction techniques for a smooth cake co-design experience.

This project will require knowledge of Unity. It will involve building 3D interaction models and running controlled studies to collect (and later analyze) data about user experiences of the developed interaction techniques.

Skills:

  • Required:
    • Programming, prototyping (C#/Unity)
  • Recommended:
    • Running user studies
    • Quantitative and qualitative analysis methods

Literature:

  • Scott W Greenwald, Alexander Kulik, André Kunert, Stephan Beck, Bernd Fröhlich, Sue Cobb, Sarah and others. 2017. Technology and applications for collaborative learning in virtual reality. Philadelphia, PA: International Society of the Learning Sciences.
  • Jan Gugenheimer, Evgeny Stemasov, Julian Frommel, and Enrico Rukzio. 2017. Sharevr: Enabling co-located experiences for virtual reality between hmd and non-hmd users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 4021–4033.
  • Hikaru Ibayashi, Yuta Sugiura, Daisuke Sakamoto, Natsuki Miyata, Mitsunori Tada, Takashi Okuma, Takeshi Kurata, Masaaki Mochimaru, and Takeo Igarashi. 2015. Dollhouse VR: a multi-view, multi-user collaborative design workspace with VR technology. In SIGGRAPH Asia 2015 Emerging Technologies. ACM, 8.
  • Yanni Mei (2020). Design a social VR tool for the remote co-design of customized cakes. Master’s Thesis. Delft University of Technology. http://resolver.tudelft.nl/uuid:78a1147b-e97b-418f-a5e6-3ce944df4f49

Designing for Serendipitous Social Encounters in Online Museum Exhibits

Contact: Alina Striner (a.striner@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Interactive exhibits in museum spaces create opportunities for serendipitous social encounters. These social encounters act as icebreakers in academic settings, allowing students and researchers to form connections with peers in a relaxed informal setting. COVID-19 has limited in-person social encounters, making it difficult for people to form such connections in the physical world.

How should we redesign interactive museum exhibits in the virtual world for serendipitous encounters? This project will redesign an interactive museum exhibit to emulate serendipitous in-person encounters, considering aspects of storytelling through 3D space design. This project will require knowledge of unity and 3D environments. The project will involve building a prototype of an interactive museum exhibit, and will involve running controlled studies to collect (and later analyze) data about user experiences in these environments.

Skills:

  • Required:
    • Experience with Unity
    • VR/AR development
    • Interest in human-computer interaction

Literature:

  • Karin Ryding. 2020. The Silent Conversation: Designing for Introspection and Social Play in Art Museums. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–10.
  • Karen Johanne Kortbek and Kaj Grønbæk. 2008. Communicating art through interactive technology: new approaches for interaction design in art museums. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges. 229–238.
  • Astrid Bin, Christina Bui, Benjamin Genchel, KaushalSali, Brian Magerko, and Jason Freeman.[n.d.]. From Museum The browser: Translating a music-driven exhibit from physical space to a web app. ([n. d.]).
  • Areti Damala, Ian Ruthven, and Eva Hornecker. 2019. The MUSETECH model: A comprehensive evaluation framework for museum technology. Journal on Computing and Cultural Heritage (JOCCH) 12, 1 (2019), 1–22
  • Jordan Graves and Brian Magerko. 2020. Community garden: designing for connectedness in online museum exhibits. In Proceedings of the 2020 ACM. Interaction Design and Children Conference: Extended Abstracts. 268–271.

Continuous Evaluation of Quality of Experience in Virtual Reality

Contact: Irene Viola (irene@cwi.nl), Abdallah El Ali (aea@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Recent advances in 3D acquisition and rendering technologies, such as low-cost sensors and cross reality (XR) devices, as well as commodity hardware with sufficient computational power, have led to a renewed interest in photo-realistic immersive virtual reality experiences, such as 360-degree videos. Such experiences are characterized by having more degrees of freedom with respect with traditional media, as the user can freely navigate and does not visualize the entire content at the same time. This may lead to different perceived quality across different parts of the 360-degree video, if they are not encoded using the same parameters. Users are generally asked to give an opinion on the overall quality of experience after they are done visualizing it, thus averaging across the variations they have seen.

The question we want to answer is: can we continuously capture the perceived visual quality of an immersive content, and how does it relate to the final user judgement? How does the continuous annotation task affect the VR experience? The project requires knowledge in Unity to develop a continuous annotation system for VR. Alternative methods for continuous annotation will be designed and tested through user studies to understand the benefits and drawbacks.

Skills:

  • Required:
    • Good Unity programming skills
    • Image and video processing basics
  • Recommended:
    • Human computer interaction
    • Design skills

Literature:

  • Mario Graf, Christian Timmerer, and Christopher Mueller. 2017. Towards Bandwidth Efficient Adaptive Streaming of Omnidirectional Video over HTTP: Design, Implementation, and Evaluation. In Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys'17). Association for Computing Machinery, New York, NY, USA, 261–271. DOI:https://doi.org/10.1145/3083187.3084016
  • BT, RECOMMENDATION ITU-R. “Methodology for the subjective assessment of the quality of television pictures.” International Telecommunication Union (2002).
  • Xue, T., Ghosh, S., Ding, G., El Ali, A., & Cesar, P. (2020, April). Designing Real-time, Continuous Emotion Annotation Techniques for 360° VR Videos. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-9).

Deep Learning-based Methods for Point Cloud Compression

Contact: Irene Viola (irene@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Point cloud representation has seen a surge of popularity in recent years, thanks to its capability to reproduce volumetric scenes in immersive scenarios. A point cloud is a collection of unordered points in 3D space. Attributes such as color and normal information are commonly associated with each point. As no connectivity information among the points needs to be stored, they result in faster rendering, thus making them suitable for real-time systems. New compression solutions for streaming of point cloud contents have been proposed and are currently being standardized by the MPEG standardization body. In the last years, several deep learning-based solutions have emerged to perform compression of point cloud contents. However, they have largely focused on encoding of the geometrical structure, and few have been proposed to tackle attribute encoding. Extending such methods to dynamic (i.e., video) sequences adds the additional complexity of considering temporal redundancy.

In this project, you will design a deep learning-based method for compression of dynamic point cloud sequences. After selecting suitable contents for training and testing, the proposed method will be compared to the state of the art to assess its performance and coding efficiency.

Skills:

  • Required:
    • Good knowledge of deep learning methods and frameworks (TensorFlow, PyTorch)
    • Image and Video Processing basics
  • Recommended:
    • Computer graphics

Literature:

  • S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, et.al. Emerging MPEG Standards for Point Cloud Compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems (IEEE JETCAS), 9(1) : pp. 133-148, 2019.
  • Quach, M., Valenzise, G., & Dufaux, F. (2020). Improved deep point cloud geometry compression. arXiv preprint arXiv:2006.09043.
  • Guarda, A. F., Rodrigues, N. M., & Pereira, F. (2020, July). Deep Learning-Based Point Cloud Geometry Coding: RD Control Through Implicit and Explicit Quantization. In 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) (pp. 1-6). IEEE.
  • Alexiou, E., Tung, K., & Ebrahimi, T. (2020, August). Towards neural network approaches for point cloud compression. In Applications of Digital Image Processing XLIII (Vol. 11510, p. 1151008). International Society for Optics and Photonics.

Objective Metrics for Point Cloud Quality Assessment

Contact: Irene Viola (irene@cwi.nl), Pablo Cesar (p.s.cesar@cwi.nl)

Volumetric data captured by state of the art capture devices, in its most primitive form, consists of a collection of points called a point cloud (PC). A point cloud consists of a set of individual 3D points. Each point, in addition to having a 3D (x, y, z) position, i.e., spatial attribute, may also contain a number of other attributes such as color, reflectance, surface normal, etc. There are no spatial connections or ordering relations specified among the individual points. When a PC signal is processed, for example undergoing lossy compression to reduce its size, it is critical to be able to quantify how well the processed signal is approximating the original one, as in the perception of the end user, which is the human being who will visualize the signal. The goal of this project is to develop a new algorithm (i.e. objective full-reference quality metric) to evaluate the perceptual fidelity of a processed PC with respect to its original version. A framework implementing the objective metrics currently available in literature to assess PC visual quality, and comparing the performance to the proposed method will also be developed. Subjective feedback on the visual quality of the signals will be collected from users to serve as ground-truth.

Skills:

  • Required:
    • Good (Matlab, Python, or C++) programming skills

Literature:

  • E. Torlig; E. Alexiou; T. Fonseca; R. de Queiroz; T. Ebrahimi, A novel methodology for quality assessment of voxelized point clouds”, 2018. SPIE Optical Engineering + Applications
  • E. Alexiou; T. Ebrahimi, “Benchmarking of objective quality metrics for colorless point clouds”, 2018 Picture Coding Symposium
  • S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, et.al. Emerging MPEG Standards for Point Cloud Compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems (IEEE JETCAS), 9(1) : pp. 133-148, 2019.
  • Meynet, G., Nehmé, Y., Digne, J., & Lavoué, G. (2020, May). PCQM: A Full-Reference Quality Metric for Colored 3D Point Clouds. In 12th International Conference on Quality of Multimedia Experience (QoMEX 2020).
  • Viola, I., Subramanyam, S., & César, P. (2020). A color-based objective quality metric for point cloud contents. In 12th International Conference on Quality of Multimedia Experience (QoMEX 2020).