The COVID-19 pandemic has changed our lifestyle and caused high demand for remote communication and experience, but current video conferencing systems do not meet basic requirements for remote collaboration due to the lack of eye contact, gaze awareness and spatial audio synchronisation. Reproduction of a real space as an audio-visual 3D model allows users to remotely experience real-time interaction in real environments, thus it can be widely utilised in various applications such as healthcare, teleconferencing, education, entertainments, etc. In this talk, a simple and practical solution to estimate geometrical structure and acoustic properties of general scenes allowing an immersive rendering of the scene to improve user experience in a virtual space. This system will address three research questions: (1) How to reconstruct the complete 3D scene geometry from a single 360 photo; (2) How to extract semantic audio and visual properties of the scene; and (3) How to adaptively render plausible audio-visual experiences to users in real-time.

Bio

Dr. Hansung Kim is an Associate Professor in the School of Electronics & Computer Science at the University of Southampton, UK. He received his BSc, MSc and Ph.D degrees in electronic and electrical engineering from Yonsei University, South Korea. He was employed as a researcher of Knowledge Science Lab (KSL) at Advanced Telecommunications Research Institute International (ATR), Japan, from 2005 to 2008, and as a senior research fellow at the Centre for Vision, Speech, and Signal Processing (CVSSP) at the University of Surrey, UK, from 2008 to 2020. His research for last 20 years includes 3D computer vision, Stereo image processing, 3D reconstruction, Augmented/Virtual Reality, Multi-modal data processing, Audio-visual data processing and Media production with over 100 published articles in 16 research projects.