Idea: Real Time Dynamic Binaural Sound

36b416ed76cbaff49c8f6b7511458883
0
poita 101 Jun 13, 2007 at 17:26

Before I go on, if you don’t know what binaural sound recording is (or just want to here how amazing it is) listen to this: http://www.3gpvids.com/forums/index.php?showtopic=481

YOU NEED YOUR HEADPHONES ON OR IT JUST SOUNDS LIKE CRAP!

Now, this is just an idea I came up with after listening to this. I really don’t have any experience with sound programming so it’s possible that this is just a wild dream with no practical or possible application. Anyway, I thought I’d share it since if it were possible then I’m sure that it would be almost revolutionary.

If you listened to the link above then I’m sure you will be amazed at how realistic it sounded and if you are like me, the first thing you thought was “How great would that sound in an atmospheric game like Doom 3 or any game for that matter!” Obviously binaural recording could be easily used for cut scenes in a game but I was wondering if it could be used dynamically during the action. Binaural recording requires the position of the sound origin relative to the listener to be known at the time of recording. Of course this is not the case if people are all around you shooting guns and constantly changing position.

Clearly to get the effect of binaural sound you would have to process the sound effects in real time during the game. This is where my idea came in.

Say we have a sound effect, say a gun-fire and we call this sound f(t) and this gunfire originated at point (x,y,z) relative to the listener. This is just the raw non-binaural sound effect. We want to make it into a fully binaural sound, that we will call g(t). I was wondering if it would be possible to get f(t) to g(t) by convoluting it with a predetermined impulse response for the given relative position.

So basically we would have:

g(t) = f(t)*h(x,y,z)
where
* is the convolution operator and
h(x,y,z) is the predetermined impulse response for the relative position of the sound that will transform f into the binaural sound g.

The problem then would be computing this in real time and also finding this impulse response function. Like I said, I don’t have any experience with sound programming so I don’t know whether performing this convolution on several sounds in real time is possible or practical.

As for finding the impulse response function. The method I had in mind was to precalculate the impulse response for many, many different input vectors (x,y,z) from all different angles and distances. This could possibly be done by doing a binaural recording of a test sound and then deconvoluting the recording to find the impulse response.

I hope that made sense. Basically the idea is to make an ordinary audio sample sound like a binaural sound by processing it in some way (convolution with an impulse response for example).

So does anyone with some knowledge in sound programming and digital signal processing know if this is achievable or not?

1 Reply

Please log in or register to post a reply.

Fd80f81596aa1cf809ceb1c2077e190b
0
rouncer 104 Jun 16, 2007 at 20:33

i dont know what binaural sound is but
ive coded a few synthesizers before, and can you calculate it per game frame
instead of per sample.

because anything you can precalculate per sample chunk, instead of per
sample speeds things up a lot.

but i skipped lessons on iir filters and copied code, so maybe i cant help you. :)