Human-Centered Multimedia Systems
We conduct research to develop and optimize multimedia systems based on the end-user’s perception, behavior and expectations in the context of use. Understanding and modeling how humans perceive multimedia signals, what are the social and communication cues that impact users’ experience when a multimedia system is used, and what are users’ expectations in the context of use of the system, is key in order to design and develop algorithms and technologies that provide a satisfactory experience to the end-user. Exploiting this knowledge, multimedia systems can be optimized to provide the best performance. For example, to optimize the bandwidth consumption for audio-visual content distribution over a mobile network, understanding the effect of the signal quality variations on user’s satisfaction can be used to optimize the transmission itself.
A key focus of our group is on human-centered distribution, compression and evaluation of multimedia signals, including research on:
- Experience-aware distribution strategies of multimedia content
- Lossy compression of visual signals, with focus on signals for immersive applications, such as Virtual and Augmented Reality
- Prototypes and infrastructures for studying multimedia distribution and user’s consumption in real-world settings
Topics
- Multimedia streaming
- Experience-Aware Networking and Software Defined Networking
- Adaptive streaming for immersive experiences
- Visual Compression
- Point cloud compression
- Lossy 360-degree image and video compression
- Capture of immersive signals
- Light-weight capture of point clouds signals
- Frame rate augmentation of dynamic point clouds
Funding
Videos
- TA2: Improving Social Communication Between Groups: https://www.youtube.com/watch?v=5XrT0f0Aw78
- Distributed Tempest: https://www.youtube.com/watch?v=GXEcJX1LIbg
- VRTogether: Ground-breaking Social Virtual Experiences: https://www.youtube.com/watch?v=dL5NX74roBg
Open Source
Publications
-
Best Demo
A Pipeline for Multiparty Volumetric Video Conferencing: Transmission of Point Clouds over Low Latency DASH. In Proceedings of the the ACM Multimedia Systems Conference (MMSys), Istanbul, Turkey, 2020.
-
Temporal Interpolation of Dynamic Digital Humans using Convolutional Neural Networks.
In Proceedings of the IEEE International Conference on Artificial Intelligence & Virtual Reality,
San Diego, CA, USA,
2019.
-
Complexity measurement and characterization of 360-degree content.
In Proceedings of the Human Vision and Electronic Imaging Conference 2019,
Burlingame, CA, USA,
2019.
-
Visual Distortions in 360-degree Videos.
IEEE Transactions on Circuits and Systems for Video Technology,
2019.
-
Emerging MPEG Standards for Point Cloud Compression.
IEEE Journal on Emerging and Selected Topics in Circuits and Systems (IEEE JETCAS),
9(1) : pp. 133-148,
2019.
-
Best Paper
Dynamic Adaptive Streaming for Multi-viewpoint Omnidirectional Videos. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys 2018), Amsterdam, NL, pp. 237-249, 2018.
-
Improving Mobile Video Quality Through Predictive Channel Quality Based Buffering.
In Proceedings of the 28th International Teletraffic Congress, (ITC 30),
Vienna, AT,
pp. 236-244,
2018.
-
Rate distortion optimized graph partitioning for omnidirectional image coding.
In Proceedings of the 26th European Signal Processing Conference (EUSIPCO),
Rome, IT,
pp. 897-901,
2018.
-
2020 IEEE Transactions on Circuits and Systems for Video Technology Best Paper Award
Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video. IEEE Transactions on Circuits and Systems for Video Technology, 27(4) : pp. 828-842, 2017.
-
Best Paper
An SDN Architecture for Privacy-friendly Network Assisted DASH. ACM Transactions on Multimedia Computing Communications and Applications (ACM TOMM), 13(3s), Article 44, 22 pages, 2017.
-
A Model for Evaluating Sharing Policies for Network-Assisted HTTP Adaptive Streaming.
Elsevier Computer Networks Journal (ComNet),
109(part 2) : pp. 234-245,
2016.
-
Enabling 3D Tele-Immersion with Live Reconstructed Mesh Geometry with Fast Mesh Compression and Linear Rateless Coding.
IEEE Transactions on Multimedia (TMM),
16(7) : pp. 1809-1820,
2014.
-
From IPTV to Synchronous Shared Experiences: Challenges in Design: Distributed Media Synchronization.
Elsevier Signal Processing: Image Communication (Elsevier IMAGE),
26(7) : pp. 370-377,
2011.
Theses
- Tong Xue, Continuous Emotion Annotation Techniques for Mixed Reality Environments, Beijing Institute of Technology, China.
- Jan Willem Kleinrouweler, Modeling and Optimization of Network Assisted Video Streaming, Vrije Universiteit Amsterdam, The Netherlands. Expected 2020.
- Mario Montagud Climent, Design, Development and Evaluation of an Adaptive RTCP-based IDMS Solution, Universidad Politecnica de Valencia, Spain. PhD Thesis, 2015.
- Ishan Vaishnavi, Coherence in Synchronous Shared Experiences, Vrije Universiteit Amsterdam, The Netherlands. PhD Thesis, 2011.
Standards
- MPEG-4.3D Graphics Compression Model
