Human-Centered Multimedia Systems
We conduct research to develop and optimize multimedia systems based on the end-user’s perception, behavior and expectations in the context of use. Understanding and modeling how humans perceive multimedia signals, what are the social and communication cues that impact users’ experience when a multimedia system is used, and what are users’ expectations in the context of use of the system, is key in order to design and develop algorithms and technologies that provide a satisfactory experience to the end-user. Exploiting this knowledge, multimedia systems can be optimized to provide the best performance. For example, to optimize the bandwidth consumption for audio-visual content distribution over a mobile network, understanding the effect of the signal quality variations on user’s satisfaction can be used to optimize the transmission itself.
A key focus of our group is on human-centered distribution, compression and evaluation of multimedia signals, including research on:
- Experience-aware distribution strategies of multimedia content
- Lossy compression of visual signals, with focus on signals for immersive applications, such as Virtual and Augmented Reality
- Prototypes and infrastructures for studying multimedia distribution and user’s consumption in real-world settings
- Multimedia streaming
- Experience-Aware Networking and Software Defined Networking
- Adaptive streaming for immersive experiences
- Visual Compression
- Point cloud compression
- Lossy 360-degree image and video compression
- Capture of immersive signals
- Light-weight capture of point clouds signals
- Frame rate augmentation of dynamic point clouds
- TA2: Improving Social Communication Between Groups: https://www.youtube.com/watch?v=5XrT0f0Aw78
- Distributed Tempest: https://www.youtube.com/watch?v=GXEcJX1LIbg
- VRTogether: Ground-breaking Social Virtual Experiences: https://www.youtube.com/watch?v=dL5NX74roBg
A Pipeline for Multiparty Volumetric Video Conferencing: Transmission of Point Clouds over Low Latency DASH. In Proceedings of the the ACM Multimedia Systems Conference (MMSys), Istanbul, Turkey, 2020.
Temporal Interpolation of Dynamic Digital Humans using Convolutional Neural Networks.
In Proceedings of the IEEE International Conference on Artificial Intelligence & Virtual Reality,
San Diego, CA, USA,
Complexity measurement and characterization of 360-degree content.
In Proceedings of the Human Vision and Electronic Imaging Conference 2019,
Burlingame, CA, USA,
Visual Distortions in 360-degree Videos.
IEEE Transactions on Circuits and Systems for Video Technology,
Emerging MPEG Standards for Point Cloud Compression.
IEEE Journal on Emerging and Selected Topics in Circuits and Systems (IEEE JETCAS),
9(1) : pp. 133-148,
Dynamic Adaptive Streaming for Multi-viewpoint Omnidirectional Videos. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys 2018), Amsterdam, NL, pp. 237-249, 2018.
Improving Mobile Video Quality Through Predictive Channel Quality Based Buffering.
In Proceedings of the 28th International Teletraffic Congress, (ITC 30),
Rate distortion optimized graph partitioning for omnidirectional image coding.
In Proceedings of the 26th European Signal Processing Conference (EUSIPCO),
2020 IEEE Transactions on Circuits and Systems for Video Technology Best Paper Award
Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video. IEEE Transactions on Circuits and Systems for Video Technology, 27(4) : pp. 828-842, 2017.
An SDN Architecture for Privacy-friendly Network Assisted DASH. ACM Transactions on Multimedia Computing Communications and Applications (ACM TOMM), 13(3s), Article 44, 22 pages, 2017.
A Model for Evaluating Sharing Policies for Network-Assisted HTTP Adaptive Streaming.
Elsevier Computer Networks Journal (ComNet),
109(part 2) : pp. 234-245,
Enabling 3D Tele-Immersion with Live Reconstructed Mesh Geometry with Fast Mesh Compression and Linear Rateless Coding.
IEEE Transactions on Multimedia (TMM),
16(7) : pp. 1809-1820,
From IPTV to Synchronous Shared Experiences: Challenges in Design: Distributed Media Synchronization.
Elsevier Signal Processing: Image Communication (Elsevier IMAGE),
26(7) : pp. 370-377,
- Tong Xue, Continuous Emotion Annotation Techniques for Mixed Reality Environments, Beijing Institute of Technology, China.
- Jan Willem Kleinrouweler, Modeling and Optimization of Network Assisted Video Streaming, Vrije Universiteit Amsterdam, The Netherlands. Expected 2020.
- Mario Montagud Climent, Design, Development and Evaluation of an Adaptive RTCP-based IDMS Solution, Universidad Politecnica de Valencia, Spain. PhD Thesis, 2015.
- Ishan Vaishnavi, Coherence in Synchronous Shared Experiences, Vrije Universiteit Amsterdam, The Netherlands. PhD Thesis, 2011.
- MPEG-4.3D Graphics Compression Model