Manipulating waveforms live in-game
Posted 13 January 2012 - 03:17 AM
I'm looking for a way to manipulate sound using code in the following manner. Basically, when one uses some sort of audio software, you are able to open an audio file and adjust several hundred different audio properties in the program via sliders, buttons, value inputs, etc. What I would like to do is have the ability to do all of that in-code. I.e., I have a "generic" audio file, and based on user input, the program can intelligently apply various quantities and degrees of all those bajillions of effects to the file.
Also, is there a way to get down 'n dirty and actually mess with the waveform itself? Basically, the goal would be to apply all the same aforementioned effects but do so by actual root-mathematical manipulation of a very generic audio file. Not really sure how to ask this, as I'm new at digi audio and don't really know how it all works. However, the main thing is, I want to be able to have code that can
intelligently edit audio on the fly as I myself would in an actual sound software interface.
Thanks for any help!
Posted 13 January 2012 - 03:56 AM
Posted 13 January 2012 - 11:16 AM
Mod files were created as a way of getting sampled sound into games without a massive memory overhead.
So they work by taking a single sample, say of a piano playing a middle C, then changing the playback rate to generate other notes.
Simple really, and very effective.
The guy who wrote the music for Cannon Fodder (was it Rob?) used his own voice for the lyrics, then upsampled it to sound like a woman.
Worth looking at for some cool techniques.
Posted 13 January 2012 - 01:36 PM
As for performance, on today's CPUs it's a walk in the park. If you want to get real creative you could even do this on the GPU at blazing speeds. With my synthesizer, I can mix two 44KHz triple oscillators in real-time on a mobile phone, or about nine 11KHz oscillators.
Posted 13 January 2012 - 04:09 PM
Posted 14 January 2012 - 10:44 AM
One is for simple, standard, mono or stereo.
The other includes a 3d vector for the position of the sound in 3d space.
In principle it is trivial to put a sound source anywhere you like. In practice it can be tricky.
The problem comes in the equation the api programmer has used for attenuation. Often you find that you have to scale the position to fit into the designed space.
However that is a problem we can help you with when you get further down the line.
Posted 15 January 2012 - 04:57 AM
Posted 15 January 2012 - 10:54 PM
Posted 16 January 2012 - 12:46 AM
There are a few things layered on that, such as having multiple versions of a sound that are randomly chosen (e.g. several different gunshot sounds for each gun, so that not all shots from that gun sound exactly alike). Certain sounds might have pitch-shifting or other adjustable parameters that can be changed by the game engine, e.g. a helicopter rotor sound that changes with speed. Some voice lines might get processed to make them sound like they're coming through a cell phone or walkie-talkie, although this can also be done as a preprocess.
There's probably very, very little direct synthesis of sounds from scratch at runtime.
Posted 16 January 2012 - 05:00 AM
Posted 16 January 2012 - 10:32 AM
There is some buffering on the music player so that you don't get drop outs when you access the DVD for data reads.
The XBOX only has 512M of ram, so you have to do something like this.
One of the things that is done a lot (and I hate when they get it wrong) is to try and link the music to the game events.
So if you are wandering through a forest you get Jethro Tull, then when you get attacked by the big bad wolf you hear Metallica.
Which is fine, and can even be useful when you don't know a wolf is coming up behind you but the music has suddenly gone metal.
But when you kick the wolf of a cliff and it's stuck 100 meters away and cannot get to you and the music is still blasting your ears because it's not dead and so you must still be in combat with it, or you are level 129,001 and a wolf only takes a blink of the eye to kill, it's annoying.
Posted 16 January 2012 - 06:16 PM
You can pull off purely synthesized sounds in a music touchpad game. Something like this. It's very easy to produce sounds that are pleasing on the ears. You could also synthesize some musical instruments fairly well. DSK Music produces a lot of free, great VST plugins that do just this.
For other gaming though, I don't think realtime synthesis would be used (not to be confused with realtime DSP, which is widely used). You could produce sounds in advanced using a synth, but it would be difficult and time consuming finding an algorithm to match the type of sound effect you're aiming for. Libraries like OpenAL and FMOD provide a nice API that allows you to plugin custom effects applied to your sounds at runtime.
Posted 16 January 2012 - 10:37 PM
Posted 27 February 2012 - 08:32 PM
They said for current AAA titles, around 50% of the non-dialog, non-music 'sounds' were something more complicated than "just play a wave file." Most often, it meant layering of multiple wav files to create one "sound." Games are also starting to use a lot of run-time DSP (digital signal processing effects, using either built-in effects in things like FMOD or WWise, or licensed from a 3rd party. (Some of the 'pro audio' dsp companies such as McDSP, WaveArts and Izotope are now licensing their algorithms to game developers).
Taking one or more wav files and processing them in real-time, tying the DSP parameters to the game physics engine can create some really great results..
And some games are have started basic physical synthesis algorithms for some sound effects-- think metallic 'bangs', wind wooshes, etc. WWise has a couple of physical modelling packages they make available.
Brian Schmidt Studios
Executive Director, GameSoundCon
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users