I’m looking for a way to manipulate sound using code in the following
manner. Basically, when one uses some sort of audio software, you are
able to open an audio file and adjust several hundred different audio
properties in the program via sliders, buttons, value inputs, etc. What
I would like to do is have the ability to do all of that in-code. I.e.,
I have a “generic” audio file, and based on user input, the program can
intelligently apply various quantities and degrees of all those
bajillions of effects to the file.
Also, is there a way to get down ‘n dirty and actually mess with the
waveform itself? Basically, the goal would be to apply all the same
aforementioned effects but do so by actual root-mathematical
manipulation of a very generic audio file. Not really sure how to ask
this, as I’m new at digi audio and don’t really know how it all works.
However, the main thing is, I want to be able to have code that can
intelligently edit audio on the fly as I myself would in an actual
sound software interface.
Thanks for any help!
Please log in or register to post a reply.
Yes, you absolutely can generate and/or process the waveform directly,
in real time. It’s a fascinating area of programming. If you’re
interested in getting your feet wet with it in a not-too-demanding
programming environment, you might try
SuperCollider. If you want to
do it more directly in C/C++, and you’re using Windows, you can use the
API (part of the Windows API). This API allows you to generate buffers
of waveform data using any algorithm you like and send them to the sound
chip to play. You can probably find some articles about it online and/or
read the docs to figure out how to make it work.
Also look at the mod file format and source code for mod players.
Mod files were created as a way of getting sampled sound into games
without a massive memory overhead.
So they work by taking a single sample, say of a piano playing a middle
C, then changing the playback rate to generate other notes.
Simple really, and very effective.
The guy who wrote the music for Cannon Fodder (was it Rob?) used his own
voice for the lyrics, then upsampled it to sound like a woman.
Worth looking at for some cool techniques.
Two other areas you can look into are
is a specification for working with digital instruments and VST (look
into version 2.X, not 3.X) is a plugin format which is basically its own
little software synthesizer, most commonly used as a soundfont, generate
effects, or filter audio input. The two are really great specs for
getting an inside look to building your own synthesizer. I based my own
synthesizer off these two and added a couple extras like ADSR envelope
and sound tracks. The possibilities are limitless.
As for performance, on today’s CPUs it’s a walk in the park. If you want
to get real creative you could even do this on the GPU at blazing
speeds. With my synthesizer, I can mix two 44KHz triple oscillators in
real-time on a mobile phone, or about nine 11KHz oscillators.
This is absolutely terrific. Thanks you all! I assume that all of these
said techniques/tools can be easily put to the task of “3D” soundspace?
I’m also assuming that that’s a function of programming (therefore it
can be done in-game, in-code).
Most sound API’s have two play functions.
One is for simple, standard, mono or stereo.
The other includes a 3d vector for the position of the sound in 3d
In principle it is trivial to put a sound source anywhere you like. In
practice it can be tricky.
The problem comes in the equation the api programmer has used for
attenuation. Often you find that you have to scale the position to fit
into the designed space.
However that is a problem we can help you with when you get further down
Thanks Stainless, very helpful there. Yes, I’ll certainly give a calling
card when I need more help! This is all great stuff to start with.
PS–how much sound is generated, these days, in top-end games? In other
words, beyond the recorded dialogue, what percentage of audio is
recorded then edited in game on the fly, versus generated completely
in-game in pure coding? I realize there’s not a standard %; I’m just
curious about how intensive the recording side and the in-game coded
audio side are in a modern-tech game.
I believe in most games the sounds are almost entirely pre-recorded, and
what happens at runtime is essentially just playing specified sounds
when an event fires (e.g. a gunshot) and mixing all the
currently-playing sounds together.
There are a few things layered on that, such as having multiple versions
of a sound that are randomly chosen (e.g. several different gunshot
sounds for each gun, so that not all shots from that gun sound exactly
alike). Certain sounds might have pitch-shifting or other adjustable
parameters that can be changed by the game engine, e.g. a helicopter
rotor sound that changes with speed. Some voice lines might get
processed to make them sound like they’re coming through a cell phone or
walkie-talkie, although this can also be done as a preprocess.
There’s probably very, very little direct synthesis of sounds from
scratch at runtime.
Hmm, I see. That’s interesting. I figured that there’d be a little more
on-the-spot synthesis. Whatever the case, I’m gonna work on that
Most of the time you get the sound effects in memory, and the music on
There is some buffering on the music player so that you don’t get drop
outs when you access the DVD for data reads.
The XBOX only has 512M of ram, so you have to do something like this.
One of the things that is done a lot (and I hate when they get it wrong)
is to try and link the music to the game events.
So if you are wandering through a forest you get Jethro Tull, then when
you get attacked by the big bad wolf you hear Metallica.
Which is fine, and can even be useful when you don’t know a wolf is
coming up behind you but the music has suddenly gone metal.
But when you kick the wolf of a cliff and it’s stuck 100 meters away and
cannot get to you and the music is still blasting your ears because it’s
not dead and so you must still be in combat with it, or you are level
129,001 and a wolf only takes a blink of the eye to kill, it’s
Synthesized sounds work great for 8-bit games. Nothing beats a good
You can pull off purely synthesized sounds in a music touchpad game.
Something like this. It’s
very easy to produce sounds that are pleasing on the ears. You could
also synthesize some musical instruments fairly well. DSK
Music produces a lot of free, great VST
plugins that do just this.
For other gaming though, I don’t think realtime synthesis would be used
(not to be confused with realtime DSP, which is widely used). You could
produce sounds in advanced using a synth, but it would be difficult and
time consuming finding an algorithm to match the type of sound effect
you’re aiming for. Libraries like OpenAL and FMOD provide a nice API
that allows you to plugin custom effects applied to your sounds at
Thanks for the input peoples, great stuff again. I figured that games do
a lot of prerecorded stuff. I myself would like to do what probably
amounts to an uncommon percentage of realtime production and synth based
on/derived from game logic which I’m developing myself. The good thing
is, the type of sound I’m looking for is quite similar to that
uber-sweet music touchpad game T.N. linked to above. Basically, I’d like
to be able to take a handful of basic tones and have a realtime effects
engine run synth on it based on said logic I’m cooking up. I don’t plan
on trying to produce the rustle of grass or the flap of wings or
something, but the soundscape and style of the world itself is very
“tone-ish”, so I think I could pull it off. Yah, doable?
I sent out a brief survey a few months ago to a mailing list with a lot
of AAA game audio developers on it..
They said for current AAA titles, around 50% of the non-dialog,
non-music ‘sounds’ were something more complicated than “just play a
wave file.” Most often, it meant layering of multiple wav files to
create one “sound.” Games are also starting to use a lot of run-time DSP
(digital signal processing effects, using either built-in effects in
things like FMOD or WWise, or licensed from a 3rd party. (Some of the
‘pro audio’ dsp companies such as McDSP, WaveArts and Izotope are now
licensing their algorithms to game developers).
Taking one or more wav files and processing them in real-time, tying
the DSP parameters to the game physics engine can create some really
And some games are have started basic physical synthesis algorithms for
some sound effects– think metallic ‘bangs’, wind wooshes, etc. WWise
has a couple of physical modelling packages they make available.
Brian SchmidtBrian Schmidt Studios
Executive Director, GameSoundCon