ViSam-PCQA: A Deep Learning-Based No-Reference Metric Guided by Saliency Information Across Three Modalities in the Projected Domain: 2D Texture, Depth, and Normal Maps.

As 3D immersive media continues to gain prominence, Point Cloud Quality Assessment (PCQA) is essential for ensuring high-quality user experiences. This paper introduces ViSam-PCQA, a no-reference PCQA metric guided by saliency information across three modalities, which facilitates the performance of the quality prediction. Firstly, we project the 3D point cloud to acquire 2D texture, depth, and normal maps. Secondly, we extract the saliency map based on the texture map and refine it with the corresponding depth map. This refined saliency map is used to weight low-level feature maps to highlight perceptually important areas in the texture channel. Thirdly, high-level features from the texture, normal, and depth maps are then processed by a Transformer to capture global and local point cloud representations across modalities. Lastly, saliency along with global and local embeddings, are concatenated and processed through a multi-task decoder to derive the final quality scores. Our experiments on the SJTU, WPC, and BASICS datasets show high Spearman rank order correlation coefficients/Pearson linear correlation coefficients of 0.953/0.962, 0.920/0.920 and 0.887/0.936 respectively, demonstrating superior performance compared to current state-of-the-art methods.

Download

An implementation of this metric can be downloaded by accessing here.

License

Copyright (c) Xuemei Zhou, Irene Viola, Ruihong Yin, Pablo Cesar. Centrum Wiskunde & Informatica (CWI).

Reference

If you wish to use any of the provided material, we kindly ask you to cite:

1. X. Zhou, I. Viola, R. Yin, and P. Cesar Visual-Saliency Guided Multi-modal Learning for No Reference Point Cloud Quality Assessment. In 3rd Workshop on QoEVMA, ACM Multimedia 2024, 2024.
   

---