I have a client interested in producing some VR video. I'm trying to get a handle on what the audio requirements and possibilities are. Does anyone have any experience or knowledge that they'd be willing to share?

Of the video's I've seen, there hasn't been much special about the sound, and my initial thought is that if it's just a static sound track (i.e. a stereo or surround full mix) tied to the video, that panning specific location dependent sounds would be a no-no since the visual perspective is dependent on the user. i.e. a visual image on the left side of someone facing forward is on the right side if the user is facing backwards.

Now today, I've stumbled across the "Cardboard SDK for Unity". I don't have any experience with Unity (yet), but it seems to me that somehow there are hooks built in that can provide perspective feedback back to the player. How is this incorporated with video? Or is it just for gaming environments?

Thanks for any insight.

-Richard

I did some audio for a 360 video. It was not easy. The scene was ambience with all sorts of people and actions happening around you, talking, clapping, yelling, etc. I had to create individual sound effects and place them around the user in the location that the user would see the object. This results in a scattered, weird, pinpointy sort of sound scape. To glue it all together I added a stereo ambience and some other general sounds so make it feel cohesive. If you're using Unity, Unity works with game objects that you can drag around and place in the world at various locations.

Good luck!

Just dipping a toe into this world, here are a couple of tools I've found that are helpful.

Two Big Ears

Oculus has a plug in SDK that works in AAX or VST.

https://developer.oculus.com/downloads/

And WaveArts Panorama looks pretty neat too, haven't played with it yet.

Wave Arts | Plugins | Panorama

We do that.
There is no standard yet. The Two Big Ears solution is working well, as are others (Two Big Ears providing the only Pro Tools based workflow that I know of yet).

The main problem to understand is that the sound field has to be shifted in real time according to the sensor input of the HMD and then dynamically rendered to virtual binaural.
If you have someone in the video in front of you, he has to come out of the center. If you turn your head, the voice has to dynamically pan as well. Other sounds as well. Again, some others not (general ambiance, music, narration etc.)

For this functionality to work, the according proprietary technology has to be implemented (via APIs) into the player/app.
So if your client has her own VR ecosystem in place, you can work with whatever 3D sound API is built into it.
If he doesn't then just stick to good old stereo. The same if he intends to publish his 360 videos on Facebook/Youtube.

As for Unity, it's one of the engines used a lot for VR experiences. Quite a lot of them are actually Unity-based apps. So you can use whatever API is available as a Unity plugin (almost all of them).

However, for pure 360/spherical videos, Unity is not a very good player choice. I know, I built one myself in Unity for testing purposes.

As you can see, with spherical videos, the main problem at this point is actually the distribution.
However, it's a super interesting topic that is changing on a daily base.