Regresar

Audible panorama: Automatic spatial audio generation for panorama imagery

Abstract:

As 360° cameras and virtual reality headsets become more popular, panorama images have become increasingly ubiquitous. While sounds are essential in delivering immersive and interactive user experiences, most panorama images, however, do not come with native audio. In this paper, we propose an automatic algorithm to augment static panorama images through realistic audio assignment. We accomplish this goal through object detection, scene classifcation, object depth estimation, and audio source placement. We built an audio fle database composed of over 500 audio fles to facilitate this process. We designed and conducted a user study to verify the efcacy of various components in our pipeline. We run our method on a large variety of panorama images of indoor and outdoor scenes. By analyzing the statistics, we learned the relative importance of these components, which can be used in prioritizing for power-sensitive time-critical tasks like mobile augmented reality (AR) applications.