CWI and the Netherlands Institute for Sound & Vision Gave a Sneak Peek into the Future of Cultural Heritage
Publication date: 2021-12-10
Imagine you are in London while your friend is in Paris - how can you visit a museum located in Amsterdam together? The MediaScape Extended Reality (XR) application might be a promising solution. From November 15th to 17th, Centrum Wiskunde & Informatica (CWI) and the Netherlands Institute of Sound and Vision gave a sneak peek into the future of shared, remote access to cultural heritage with the demonstration of a pioneering application, MediaScape XR.
MediaScape XR enables visitors to interact with the heritage artefacts of Sound and Vision in a virtual space shared with other remote visitors, while allowing social interactions in real time between them. The demonstration took place during the VRDays - Immersive Tech Week - in OBA Oosterdok Amsterdam. The virtual demo was open to all visitors of OBA Oosterdok for free, and was experienced and evaluated by around 150 visitors.
What is the MediaScape XR experience?
With MediaScape XR, visitors can freely walk through the virtual 3D museum of Sound and Vision, and specifically, interact with the 3D recreation of a single cultural artefact: a costume once worn by Jerney Kaagman, the frontwoman of the Dutch rock band Earth and Fire, for the performance of their song “Weekend” on the television program TopPop in 1979. The whole experience is divided into two main parts: touring the museum and reliving the history. In the virtual museum, visitors could freely observe the 3D costume from different perspectives and scales, and read the anecdotal stories. After that, they would take a “time travel” to the costume’s history, that is, the TopPop show in 1979. Immersed in the historical setting, visitors could try on the shiny blue suit and recreate the fantastic musical show in their own style.
Research and Design Process
The MediaScape XR experience was designed with a user-centric approach, collaborating with curators from Sound and Vision as well as museum lovers. We conducted focus group sessions and co-creative workshops with them to understand the stakeholders’ motivations, gather design requirements and collect novel ideas.
By interviewing curators, it was decided to focus on one specific artefact, the costume. This artefact is firmly connected to the country’s cultural memory and hence brought more popularity and better storytelling. Moreover, it allows for more immersive and fun experiences in VR, such as trying on the suit. A list of design requirements was identified from the research results, while also the interaction and the user experience flow were decided. Following our findings, the team created the digital 3D reproductions of the artefact, museum, TopPop musical hall, as well as other exhibit elements. In addition, it was decided what kind of guide information and educational content will be shared with the visitors. We wanted to make sure visitors understand the story behind the artefact, technology and the context of this experience.
Technical infrastructure and setup for VRDays
Mediascape XR is a SocialVR application built on top of the VRTogether platform that allows us to connect multiple users in the same virtual space. The application, developed in Unity3D engine, allows this multi-user setup. To capture the users in real time, we use the cwipc framework. This framework consists of a set of libraries and tools capable of generating a full pipeline for volumetric communication between multiple clients. It performs capturing, encoding, transmission, decoding and rendering of point clouds in real time.
The specific setup we used to showcase the system at VRDays consisted of two users. Each user was captured by three 3D sensors and was wearing an HTC VIVE PRO headset with controllers assisting them to navigate and interact in the virtual space. Although for the demo at OBA two users were placed side by side, it is not a local network communication, meaning that the setup can also work from 2 remote locations (for example one user in Paris and the other in London).
Vision
Compared with the traditional way to access museum archives such as websites, MediaScape XR offers a novel experience, immersing visitors in the environment and enabling them to have a close-up observation and interaction with the artefact. In MediaScape XR, gamification, education, and socialisation are well integrated. The ultimate goal of MediaScape XR is not only to digitise museum artefacts for virtual 3D worlds, but to make them more accessible to a wider dislocated audience, with visitors being able to interact in real-time with each other, while breaking the 2D screen We hope that through these types of technologies, we might be able to encourage everyone to visit museums together, independently of their location, more frequently.
More Information:
- YouTube Videos
- https://youtu.be/zzB7B6EAU9c: long version
- https://youtu.be/I7kY1cMZyD0: short version
- Other demonstrations on social VR by DIS
- Cutting the CWI-75th birthday cake in virtual reality (2021): https://youtu.be/KcRpp0s50RQ
- Remote consultation of a doctor after an accident (Demo at VRDays 2020): https://youtu.be/EuoRfy18_Fc
- Demo of a novel Social VR movie (part of the VRTogether project, 2017-2020): https://youtu.be/0l82gTc1WCQ
- Social VR pipeline (best demo award at ACM Multimedia Systems 2020): https://doi.org/10.1145/3339825.3393578
- Related news articles
