Follow us on: GitHub YouTube Vimeo Twitter

Human-Centered Multimedia Systems

We conduct research to develop and optimize multimedia systems based on the end-user’s perception, behavior and expectations in the context of use. Understanding and modeling how humans perceive multimedia signals, what are the social and communication cues that impact users’ experience when a multimedia system is used, and what are users’ expectations in the context of use of the system, is key in order to design and develop algorithms and technologies that provide a satisfactory experience to the end-user. Exploiting this knowledge, multimedia systems can be optimized to provide the best performance. For example, to optimize the bandwidth consumption for audio-visual content distribution over a mobile network, understanding the effect of the signal quality variations on user’s satisfaction can be used to optimize the transmission itself.

A key focus of our group is on human-centered distribution, compression and evaluation of multimedia signals, including research on:

  • Experience-aware distribution strategies of multimedia content
  • Lossy compression of visual signals, with focus on signals for immersive applications, such as Virtual and Augmented Reality
  • Prototypes and infrastructures for studying multimedia distribution and user’s consumption in real-world settings

Topics

  • Multimedia streaming
  • Experience-Aware Networking and Software Defined Networking
  • Adaptive streaming for immersive experiences
  • Visual Compression
  • Point cloud compression
  • Lossy 360-degree image and video compression
  • Capture of immersive signals
  • Light-weight capture of point clouds signals
  • Frame rate augmentation of dynamic point clouds

Funding

Videos

Open Source

  • MP3DG: Development framework for point cloud compression algorithms
  • MDCS

Publications

  • Best Demo
    J. Jansen, S. Subramanyam, R. Bouqueau, G. Cernigliaro, M. Martos Cabré, F. Pérez, and P. Cesar A Pipeline for Multiparty Volumetric Video Conferencing: Transmission of Point Clouds over Low Latency DASH. In Proceedings of the the ACM Multimedia Systems Conference (MMSys), Istanbul, Turkey, 2020.
  • I. Viola, J. Mulder, F. De Simone, and P. Cesar Temporal Interpolation of Dynamic Digital Humans using Convolutional Neural Networks. In Proceedings of the IEEE International Conference on Artificial Intelligence & Virtual Reality, San Diego, CA, USA, 2019.
  • F. De Simone, J. Gutiérrez, P. Le Callet Complexity measurement and characterization of 360-degree content. In Proceedings of the Human Vision and Electronic Imaging Conference 2019, Burlingame, CA, USA, 2019.
  • R.G. de A. Azevedo, N. Birkbeck, F. De Simone, I. Janatra, B. Adsumilli, P. Frossard Visual Distortions in 360-degree Videos. IEEE Transactions on Circuits and Systems for Video Technology, 2019.
  • S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, et.al. Emerging MPEG Standards for Point Cloud Compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems (IEEE JETCAS), 9(1) : pp. 133-148, 2019.
  • Best Paper
    X. Corbillon, F. De Simone, G. Simon, and P. Frossard Dynamic Adaptive Streaming for Multi-viewpoint Omnidirectional Videos. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys 2018), Amsterdam, NL, pp. 237-249, 2018.
  • J.W. Kleinrouweler, B. Meixner, J.W. Bosman, J.L. van den Berg, and P. Cesar Improving Mobile Video Quality Through Predictive Channel Quality Based Buffering. In Proceedings of the 28th International Teletraffic Congress, (ITC 30), Vienna, AT, pp. 236-244, 2018.
  • M. Rizkallah, F. De Simone, T. Maugey, C. Guillemot, P. Frossard Rate distortion optimized graph partitioning for omnidirectional image coding. In Proceedings of the 26th European Signal Processing Conference (EUSIPCO), Rome, IT, pp. 897-901, 2018.
  • 2020 IEEE Transactions on Circuits and Systems for Video Technology Best Paper Award
    R. Mekuria, K. Blom, and P. Cesar Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video. IEEE Transactions on Circuits and Systems for Video Technology, 27(4) : pp. 828-842, 2017.
  • Best Paper
    J.W. Kleinrouweler, S. Cabrero, and P. Cesar An SDN Architecture for Privacy-friendly Network Assisted DASH. ACM Transactions on Multimedia Computing Communications and Applications (ACM TOMM), 13(3s), Article 44, 22 pages, 2017.
  • J.W. Kleinrouweler, S. Cabrero, R. van der D. Mei, and P. Cesar A Model for Evaluating Sharing Policies for Network-Assisted HTTP Adaptive Streaming. Elsevier Computer Networks Journal (ComNet), 109(part 2) : pp. 234-245, 2016.
  • R. Mekuria, M. Sanna, E. Izquierdo, D.C.A. Bulterman, and P. Cesar Enabling 3D Tele-Immersion with Live Reconstructed Mesh Geometry with Fast Mesh Compression and Linear Rateless Coding. IEEE Transactions on Multimedia (TMM), 16(7) : pp. 1809-1820, 2014.
  • I. Vaishnavi, P. Cesar, D.C.A. Bulterman, O. Friedrich, S. Gunkel, and D. Geerts From IPTV to Synchronous Shared Experiences: Challenges in Design: Distributed Media Synchronization. Elsevier Signal Processing: Image Communication (Elsevier IMAGE), 26(7) : pp. 370-377, 2011.

Theses

  • Tong Xue, Continuous Emotion Annotation Techniques for Mixed Reality Environments, Beijing Institute of Technology, China.
  • Jan Willem Kleinrouweler, Modeling and Optimization of Network Assisted Video Streaming, Vrije Universiteit Amsterdam, The Netherlands. Expected 2020.
  • Mario Montagud Climent, Design, Development and Evaluation of an Adaptive RTCP-based IDMS Solution, Universidad Politecnica de Valencia, Spain. PhD Thesis, 2015.
  • Ishan Vaishnavi, Coherence in Synchronous Shared Experiences, Vrije Universiteit Amsterdam, The Netherlands. PhD Thesis, 2011.

Standards

  • MPEG-4.3D Graphics Compression Model