User-centered Optimization for Immersive Multimedia Systems
NWO WISE Grant
Starting summer 2021
The project will focus on optimizing transmission and rendering of immersive media contents. In particular, the focus is on designing and integrating novel algorithms to improve compression and transmission efficiency of immersive media contents through Quality of Experience (QoE) optimization. We want to explore how to maximize the QoE for the user, while adhering to the technological constraints. Such optimization can be performed at the acquisition, transmission and rendering stage.
June 2020 - ongoing
MediaScape is a Public-Private project, together with Sound and Vision (Nederlands Instituut voor Beeld en Geluid), researching innovative interaction technologies for providing enriched access to cultural collections. MediaScape will explore novel mechanisms for making audiovisual content more approachable, accessible, and understandable in context. Instead of generic application and interfaces, it will design and implement an innovative infrastructure that can better realise the full potential of cultural heritage collections. In particular, the project will bridge between the audience and the collection, through an immersive exhibition where visitors can directly interact with the digital items. The project will last for 18 months and the consortium includes CWI, het Nederlands Instituut voor Beeld en Geluid, and Dropstuff.
January 2020 - ongoing
Opera uses all the visual and performing arts to create extraordinary worlds of passion and sensibility. It is rightly recognized as a great achievement of European culture. And yet a form that once inspired social and artistic revolutions is often seen as the staid preserve of the elite. With rising inequality and social exclusion, many see opera—if they think of it at all—as symbolic of what is wrong in Europe today. TRACTION aims to change that using opera as a path for social and cultural inclusion, making it once again a force for radical transformation. We do not want to make opera palatable to those who don’t attend. We want to define new forms of artistic creation through which the most marginalized groups (migrants, the rural poor, young offenders and others) can work with artists to tell the stories that matter now. By combining best practice in participatory art with digital technology’s innovations of language, form and process, we will define new approaches to co-creation and innovate in three fields: a) Opera creation and production; b) Immersive and interactive digital media; and c) Social integration and community development. Experimental projects in inner-city Barcelona (ES), a youth prison in Leiria (PT) and rural communities in Ireland will test and share new ideas.
Affective Computing in the Automotive Domain
July 2019 - ongoing
This two-year project will explore the future of cars and interactive systems, with a focus on creating intelligent and emotion-aware systems that can contribute to better automotive user interfaces. This joint-industrial project aim to produce research that is working on real-world problems in the automotive domain. This project has a strong HCI component where we are working with users in a driving simulator and/or test track setting, physical prototyping with physiological sensors, designing and running controlled simulator) studies. Apart from that, the project also involves areas such as signal processing and machine learning, drawing upon recent advances in machine learning for affective and attentional state classification and/or regression using image and physiological data.
Joint PhD Program with Xinhuanet
July 2018 - ongoing
Centrum Wiskunde & Informatica (CWI) has launched a joint industrial PhD program with Xinhuanet in the research area of Social Internet of Things (IoT), smart textiles, and physiological computing. Xinhuanet Co. Ltd. is a key website in China with global influence, sponsored by Xinhua News Agency, the Chinese national news agency. The Distributed and Interactive Systems (DIS) group, together with Xinhuanet, has hired a full-time PhD student, who will work together with Dr. Pablo Cesar and Dr. Abdallah El Ali, and with the Future Media Convergence Institute at Xinhuanet. The program provides the unique opportunity for conducting research in a top research institute in Europe, based on real-world problems related to healthcare or to the creative industries. This project additionally strengthens bilateral relationships between CWI and Xinhuanet.
An end-to-end system for the production and delivery of photorealistic social immersive virtual reality experiences - VRTogether [EU H2020-ICT-19-2017 Project]
October 2017 - January 2021
VR-Together will offer new ground-breaking virtual reality experiences based on social photorealistic immersive content. For this purpose, it will develop and assemble an end-to-end pipeline integrating state-of-the-art technologies and off-the-shelf components. The challenge of VR-Together is to create photorealistic truly social virtual reality experiences in a cost effective manner. Immersive media production and delivery through innovative capture, encoding, delivery and rendering technologies. It will introduce new methods for social VR evaluation and quantitative platform benchmarking for both live and interactive content production, thus providing production and delivery solutions with significant commercial value
W3C Standardization Activities
January 2010 - January 2021
CWI has worked with the World Wide Web Consortium (W3C) ever since its establishment in 1994. The presence of researchers as well as R&D projects in the development of Web technologies and standards has always been seen as essential in the development of the Web; CWI is proud to have contributed actively to this evolution. Researchers from the CWI have been active in the past in the development of technologies and standards such as HTML, CSS, SMIL, RDF, RDFa, XForms, OWL, R2RML, JSON-LD, and SPARQL. Beyond the participation in the relevant Working Groups, representatives of the DIS group also acted as Working Group (co-)chairs, as well as activity leads for HTML and the Semantic Web as part of the core staff of W3C. The current activities at W3C concentrate, on the one hand, on the further development of the XForms standard and, on the other hand, taking the technical leadership role in the Publishing@W3C Activity at the W3C. Finally, work is also continuing in areas related to data on the Web, in particular in the latest developments around JSON-LD.
Delivering Single & Multi-Screen Content Services for Immersive, Customised and Shared Experiences in Homes and Social Spaces - 2-IMMERSE [EU H2020-ICT-2015]
December 2015 - December 2018
Surveys show that although around 80% of people are using a second device (phone, tablet or laptop) when watching TV only 20% of them are engaging with ‘companion content’. The 2-IMMERSE project will innovate around the delivery of experiences that are created to be multi-screen in production yet delivered to be flexible across single and multi-screens and responsive to the preferences of individual audience members. We will build and trial services that delivery multi-screen experiences of high value content including theatre, Grand Prix motorcycle racing and live professional football in the home, in school, and in public venues.
Project Cairo: an intelligent soft-robotic jacket
NWO Humanities - Knowledge Innovation Mapping (KIEM)
October 2017 - March 2018
Project Cairo is a joint effort between CWI, AMOLF, and the Dutch textile designer SME ByBorre to create an urban jacket that pushes the state-of-the-art of smart-textiles. The jacket will sense and reason about the wearer activity, profile and external conditions to trigger changes in its structure and insulation. To build it, we will exploit new materials developed by AMOLF, i.e. soft-robotic matter, as a way of changing the physical properties of textiles. This will be added to the proven expertise in merging sensor technology and artificial intelligence, developed by CWI, with the innovative textile design and fabrication techniques of ByBorre.
Personalized Aftermovies and Festival Experiences
NWO - Creative industry - Knowledge Innovation Mapping (KIEM)
January 2017 - December 2017
A multidisciplinary and collaborative project with the Lectoraat CrossMedia Hogeschool van Amsterdam, Fonk Mobile and Festival deBechaving. The project looks into the technical challenges and the appropiate bussiness models to enrich the experience of festival visitors. In specific, it focuses on the impact of after movies.
An exploration of the future of club culture. ByBorre Private-Public Partnership.
August 2016 - October 2016
As part of the Amsterdam Dance Event the Distribute and Interactive of CWI was asked to utilize technology to explore how the club of the future could react to the levels of excitement of the clubbers. The group developed a system that could capture the energy levels of the crowd and feed them in real time to a visualization engine as well as alter the stimuli of a room to match the activity levels of the crowd. Furthermore, the logged data was summarized in personal experience summary artifacts, delivered to the guests after the event.
Xinhuanet Private-Public Partnership
October 2014 - September 2016
CWI will be helping Xinhuanet to build the first advanced user experience laboratory in China. Both institutions will explore the use of sensor technology to better assess the user experience during media consumption. Offering rich, personalized media content is a major research topic in the creative industry. The Distributed and Interactive Systems (DIS) group at CWI, a major international player on user experience research, will help improve Xinhuanet’s media offerings. Traditional evaluation methodologies like questionnaires do not always lead to accurate and reliable results. By using biosensors, like skin conductance sensors, the level of attention of the users can be reliably measured. Collecting and analyzing this data provide Xinhua with valuable insights for applications ranging from media content to remote teaching and precision marketing.
REal and Virtual Engagement in Realistic Immersive Environments - Reverie [EU FP7 IP Project]
September 2011 - June 2015
REVERIE revolutionizes the way of immersive media distribution from a passive centralized context to a personalized highly distributed framework. The research outcome of REVERIE will provide for the individuals new ways of 3D/immersive media sharing and distribution under a socially aware, personalized, collaborative and distributed framework. REVERIE maintains the quality of media content and offers to the individual’s high definition immersive media experiences under, however, an optimized end‐to‐end approach. These research outcomes will strengthen the positioning of European Industries especially SME’s in the area of immersive and interactive multimedia content / services delivery. Within REVERIE, we believe that the time is ripe to push social networking towards the next logical step in its evolution: to immersive collaborative environments that support realistic inter‐personal communication.
Video Communications for Networked Communities - Vconect [EU FP7 STREP Project]
December 2011 - November 2014
The Vconect vision is the adoption of high-quality enriched video as a medium for mass communication within communities. Vconect is building a video communication platform which models and supports the complex communication topologies that characterize conversations between group members. The system will take intelligent decisions to mediate the communication at the level of audio-visual choices, screen layout and network capabilities. Vconect is ensuring the wide applicability of the platform by implementing, testing and evaluating it in the context of two different use cases. The first based on the integration of video conferencing into social networking services, the second based on group mediated performance.
February 2008 - March 2012
The TA2 vision: Making communications and engagement easier among groups of people separated in space and time. How can technology help to nurture family-to-family relationships? This was the question asked by the collaborative project “Together Anywhere, Together Anytime”(TA2). For many people, families form the key social unit. Many of our enduring experiences, holidays, celebrations and moments of fun and laughter are framed as family events. This is something that current technology does not address well: modern media and communications serve individuals best, with phones, computers and electronic games devices tending to be individually owned and providing individual experiences. TA2 tried to redress this imbalance, by exploring how technology can support group to group communication.
Interactive Networked Experiences in Multimedia for You - iNEM4U [EU FP7 STREP Project]
January 2008 - December 2009
In a perfect world, networked electronic media enable individuals and groups of users to share their interactive media experiences in an intuitive and seamless manner, regardless of their choice of device and networks. iNEM4U will design, prototype and evaluate a distributed service infrastructure that supports these goals. To accomplish this, the project will extend and combine existing services from different technology domains, such as Web 2.0, IMS, peer-to-peer and IPTV services. In particular, iNEM4U will investigate innovative solutions in three areas: new generic services that facilitate cross-domain interoperability (identity management, media synchronization and metadata integration), user-centric services for enhanced shared media experiences (context-dependent cross-domain recommendation system, community management), and business models that provide benefits for the entire content distribution value chain.
BRICKS PD-3 [NWO Project]
November 2005 - June 2009
In our current and future knowledge society, ICT plays a key role. Examples of important developments already omnipresent in society are fixed and mobile networks, distributed ubiquitous computing, personalized information systems with context awareness, ambient intelligence systems, etc. These developments are driven by high-level, innovative research which in turn generates new challenging research questions addressing new issues of economic and societal relevance; for example, increased attention for security, dependability, authentication, identification and privacy protection. Innovations in the ICT sector lead to new products and services and thus to economic activities and wealth creation. ICT improves products, services and labor productivity in many different sectors: industry, commerce, transportation, health care, education, safety, and the environment. BRICKS is addressing the need for a strong impulse in fundamental research in informatics. This impulse is necessary for improving the innovation climate in the Netherlands and for generating sustainable economic growth driven by ICT.
Service Platform for Innovative Communication Environment - SPICE [EU FP6 IP Project]
March 2006 - July 2008
SPICE (Service Platform for Innovative Communication Environment)is addressing the still unsolved problem of designing, developing and putting into operation efficient and innovative mobile Service creation/execution platforms for networks beyond 3G. This project will research, prototype and evaluate an extendable overlay architecture and framework to support, easy and quick service creation, test and deployment of intelligent mobile communication and information services. Building on significant advances in IT technologies, the SPICE platform will support multiple heterogeneous execution platforms allowing for new, innovative services to be spread across different operator domains and over different countries realizing a variety of business models.
Passepartout [EU EUREKA ITEA Project]
November 2005 - June 2007
PASSEPARTOUT focused on bringing tailored high-definition (HD) media content into the lives of the family. Results not only help to make sense of the input available but also to select, filter, scale and schedule HD content for presentation in the best possible way in the networked home on multiple large and small display devices. Work included seamless integration into home media centers and networks, with smart graphic displays controlling systems operation and interactive contents packaging.