Hi Has anyone got much experience in outputting audio for Samsung Gear VR or Oculus.

My first step is to try to specialise a 5.1 or 7.1 and exporting it for Samsun Gear VR initially. This is to test head tracking. I can't find anything yet on the web and need some pincers

Cheers

How do you plan on delivering the content to the Samsung Gear or Oculus? Generally, it will be an app, or an engine to create an app that can respond to the head tracking. These apps have their own audio engines, so you will probably need to find the one you want to use and research up on that.

Often the samsung and the oculus use the exact same library/vr sound system. Normally it's wwise or directly with the unity sound stuff. If it's more "internet" based then it's tags like in mozvr (basically x,y sound locations from mono tracks). It can go from fairly easy to massively complex, depending on the package and what people want to do. There aren't any solid standards being used at volume aside from using the various libraries. Cinematic VR is btw a lot different then "game" vr, lot easier to move 5.1 to cinmatic VR (like 360 3d streams, since the "location" is normally not modifiable by the user).

This is a complicated topic.
First of all: Using channel-based 5.1 or 7.1 won't work for immersive audio with headtracking.
At all.
Not even using 5.1 beds in the mix works really well.
In order to support dynamic panning based on headtracking, this has to be implemented in the distribution/playback environment.
Meaning, you need a dedicated player app for that.

F.e. you can license the technology from TwoBigEars, put it via API in your VR player app, then use their Spatial Worstation platform to mix to that proprietary format in PT (or use Unity f.e.).

There are others with similar solutions (Fraunhofer Cingo, Sennheiser is developming something right now, I'm meeting with Dolby later this week to see what they are doing), but taking an exising channel-based mix and 'magically' have it follow your head movement is not possible.

I don't agree that sound for Cinematic VR is easier to implement than CGI-based ('game') VR, but more the other way 'round.
It also has to implement dynamic panning regarding to headtrack data, and sticking a sound source object to a game object and have it move through a 3d space is a lot easier in f.e. Unity than creating that movement in let's say Protools.

We experimented with attaching motion trackers to objects in normal movies, exporting the xyz keyframes and transferring them into PT automation data, but this is not really easy to do.

Im researching wwise and fmod. I had read somewhere that a 5.1 mix did rotate with head movement . The other option is to create a binaural mix so at least it sounds in surround on headphones. The client only wants to deal with a vid file with audio embedded. I am really at the very start of this research and the learning curve is steep.
Apparently latest oculus software for Rift and Gear VR will move a 5.1 mix - just seeing if anyone has done this?
Next step is wwise. But even then it seems that a 5.1/7.1 bed is useful as dsp resources can't manage unlimited objects

Is the video file static though? I mean, is it locked in its movement, or does it track head movement in real time as the viewer moves his head? I'm just asking since you said "vid file with audio embedded". I mean, to me it read a bit like they wanted a 3d environment for a pre-recorded "locked" video....

If it's not though, couldn't you possibly mix 5.1 and treat each channel as an individual object, and then lock/track them in 3d space?

On the 5.1, it's not "exactly" normal 5.1, I don't do it like a "normal" 5.1, but here are things I'm experimenting with. For the L/R/LR/RR I keep the eq and levels the same (i.e. have all of the sound levels the same, also don't adjust the eq on the LR &RR like you would normally do with 5.1). The reason for this is when you rotate the LR it might become the R, so you want even eq and levels. For the C I think of this more as a omni directional/mono, same with the ".1". So it's not the center between L & R, but is really omni directional (i.e. i think of it as a spherical mic). Also I'm extremely careful of adding reverb, though not perfect, most of the libraries do the reverb (and often the music ducking). Just have fun and don't worry about it, really "mono"/omni is your friend. Also give a look at the new jungle book 3d trailer (of the snake), it's probably more of what your being asked to do. There is a huge variation in what people mean by VR, I'd start off with 360 video just to get the hang of it (this is what the disney clip is doing). Disney Launches ‘Jungle Book’ VR Experience | Animation World Network I find it somewhat random getting the jungle book thing running, just try it a bunch of different things(phones/computers).

If someone on gearslutz knows, would be real nice to get a run down of the sound for jungle book, both in the film and the vr stuff.

I btw also get nice surround sound (4 mic) recordings, it's real hard to just take a "film" recording and make it so you can rotate it without it getting phasy.

it does pan and follow head movement. Im aware that levels and eq will need to stay constant. But in reality cinema processors handle all that and I have no hard and fast rules for specific sound eq on rears. there is so little out there for developers it seems. The final destination is to Samsung Gear VR. I am going to do two tests..one with a 6 track atmos and the other converted to binaural. Neither will pan but at least the binaural one will give a sense of immersion. From this learning I will hopefully be able to work out the lengths that they wish to go to

Hey Chris,

I'm pretty new to VR as well and Kosmok. and others have already given experienced advice, but you might want to look at this:
3Dception Spatial Workstation by Two Big Ears

This has been mentioned here before and they are great for support - might be worth dropping them a line and you could actually do the mix etc. in PT. As far as I understand...

Cheers for all the help guys. Def going to hit up two big ears guys

Chris

For the TwoBigEars (and any of the other 'VR audio pipelines') your client has to invest into licensing their technology into a player app.

there's no way to have a static channel-based mix follow head movement that I know of without a dedicated player app.

FWIW, today I listened to what Dolby has in the pipeline, and so far it sounds convincing.