서강대학교

초록

The use of multimedia for communicating information by producing, transmitting, and consuming interactive contents is widely popular among general viewers. Recent improvements in technology have resulted in the use of improved quality of multimedia contents being consumed at higher levels of immersion with the help of Virtual Reality (VR) headsets or immersive displays. Moreover, the recent increase in the use of interactive 3D models, in the form of point clouds or textured meshes, provides an interactive solution to modelling real world 3D information from 2D images. The existing interactive multimedia creation, consumption and 3D modelling solutions generally perform well. However, these solutions are lacking in terms of geometric and photometric stitching quality, scalability, completeness, accuracy, and robustness. In this work, different methods are proposed to improve upon existing state-of-the-art (SOTA) methods. In the first section (Chapter 2), we provide a novel and robust framework for stitching and rendering of VR contents in an end-to-end manner. The VR framework provides a scalable solution for calibrating multiple cameras in a rig using the proposed on-the-go camera calibration routines. The frames from calibrated cameras are stitched and rendered on different types of display devices in real-time. In the second section (Chapter 3), a qualitative analysis of the proposed VR framework is performed for different display devices and calibration configurations. In the qualitative analysis, the levels of difficulty and presence, as experienced by the users under different settings, are measured and compared. In the third section (Chapter 4), we propose the use of Atrous Spatial Pyramid Pooling (ASPP) layers and 3-dimensional training loss function in existing deep learning based depth estimation networks. The depth inferred from the proposed depth estimation networks is processed to create improved 3D models of real-world objects. The proposed methods provide significant qualitative and quantitative improvements as compared to existing SOTA methods. The end-to-end VR framework provides SOTA VR experience with currently the best geometric alignment among existing solutions. Moreover, the use of proposed methods for depth estimation provides improved completeness and overall scores as compared to existing solutions on DTU MVS data set.