Finally! . . .a reasonable discussion on recording a 3D image.

Cheers, that's my evening viewing sorted!

Will report back...

thx ray - interesting take on the height channels; gotta try this on the next opportunity (which i hope is coming up shortly)!



[as far as the spacing of the horizontal 'main' array is concerned, excellent lf articulation doesn't comes as a surprise, at least not to me: i've mentioned in many threads the benefits of arraying mics in similar ways as subwoofers - you can take it even a step further and 'tune' the array to the room's lf behaviour! maybe i should get in touch with schoeps one of these days...]

No maybe to it, ddy!

Right finally watched that.

Very interesting yes. I use a few vertical facing mics on organ so happy to hear that story.

Ordering more Schoeps...still not enough for giant 3d square (one day.)

One thing the prof hinted at was recording for unknown future formats. I suspect in not too long you will be able to sit in the best ever concert by your favourite band from history while at home.

Having material captured in a 3d format will help with this.

Does this imply upscaling of stereo or mono to some sort of multichannel format?

I have seen some prototype pseudo AI based upscaling to immersive multichannel.

Early days but fairly obvs this is where its going.

I'd expect 3-4 years to be a commercial product, interestingly music was not high on their list of applications!

I'll happy trade that box of electrics for even more Schoeps.

Perhaps a dozen years back I noticed (the trivial but surprisingly effective) Kurtosis Maximization Independent Component Analysis (ICA) being applied to separate sources within the blended signals of mic arrays. Additional source separation (linear algebra) algorithms quickly followed and found their way into commercial products.

Now with the success of diffusion models in AI for image generators and (more recently) DLLMs, I wouldn't expect viable commercial products to still be still 3-4 years out?

Diffusion models wouldn't bring us true upscaling, per se; but (done right) it would likely sound like it.

My current bet anyway. . .

Hmmn Kurtosis Maximization Independent Component Analysis I do not know.

Off to read, may be some time.

Example MATLAB code is below [1]. But, the approach requires a separate mic in the array for each signal source we want separated, and it’s not particularly suitable for music production—as we are separating sources. . .and are not chasing aesthetic room acoustics.

More directly promising to me is smart application of diffusion models. Here Karpathy mentions recent progress in LLMs. . .

https://x.com/karpathy/status/1894923254864978091

Of course, what I have in mind for upscaling channels is a significantly different workflow.


[1] MATLAB Kurtosis Maximization ICA :
% Basically, we whiten the data to second order. This makes the unmixing matrix always orthogonal.
% Next we map the fourth order statistics down into a matrix such that the inner
% product of a unit vector with this matrix gives (roughly) the kurtosis in that direction.
% The maximal eigenvectors of this new matrix are the (orthogonal) directions of maximum kurtosis
% which for supergaussian sources are pretty good guesses at the unmixing directions.
%
[x1, Fs1] = audioread('mix1.wav');
[x2, Fs2] = audioread('mix2.wav');
xx = [x1, x2]';
yy = sqrtm(inv(cov(xx')))*(xx-repmat(mean(xx,2),1,size(xx,2)));
[W,s,v] = svd((repmat(sum(yy.*yy,1),size(yy,1),1).*yy)*yy');
a = W*xx; %W is unmixing matrix
subplot(2,2,1); plot(x1); title('mixed audio - mic 1');
subplot(2,2,2); plot(x2); title('mixed audio - mic 2');
subplot(2,2,3); plot(a(1,, 'g'); title('unmixed wave 1');
subplot(2,2,4); plot(a(2,,'r'); title('unmixed wave 2');
audiowrite('unmixed1.wav', a(1,, Fs1);
audiowrite('unmixed2.wav', a(2,, Fs1);

Note : smiles happily inserted by Gearspace over colon close parentheses.

Attribution EDIT: The code sample came from Yair Weiss and Eero Simoncelli. However, the URL where I originally found it published is no longer publicly accessible.

Matlab is amazing in its productivity per line of code. So much going on in that script.

Indeed, all of the heavy lifting is done in just two lines of code:

yy = sqrtm(inv(cov(xx')))*(xx-repmat(mean(xx,2),1,size(xx,2)));
[W,s,v] = svd((repmat(sum(yy.*yy,1),size(yy,1),1).*yy)*yy');