Basic Audio

Audio is one part of gamedev where I have very little proper low level experience. I know there is a audio buffer that needs filling, and needs filling on time. To do that I’ve relied on libraries like SDL_Mixer or rodio. So that’s the perspective I’ll be tackling web audio from - not replicating the byte shuffling, but the next level of abstraction.

Fortunately, as I’ve found many times so far, there is a modern audio API available - WebAudio.

Play Sound Effect

Audio Element

The simplest way to play a clip doesn’t actually need the WebAudio API. I can just use the audio element. Much like the img tag loads and displays images, it loads and plays audio files:

Demo

<audio id="clip"></audio>
<button id="play" onclick="play()" disabled>Play</button>

let clipElement;

window.addEventListener('DOMContentLoaded', (event) => {
    clipElement = document.getElementById('clip')

    clipElement.addEventListener('canplaythrough', () => {
        document.getElementById('play').removeAttribute('disabled');
    })

    clipElement.src = 'soundeffect.wav';
});

function play() {
    clipElement.play();
}

The audio element is created without a reference to a sound file, I set that after I’ve added an event listener to the canplaythrough event. That event fires when the audio file has finished being loaded. The play button is then enabled.

Of note is the beahvior when hitting the play button as fast as I can. The clip neither plays multiple times over itself nor does it restart the clip. The audio element will play the clip through, ignoring calls to play if it’s currently playing.

This means if I want to allow multiple simultaneous instances of the clip to play I’d need multiple audio elements.

WebAudio

WebAudio is a bit more involved, but allows for quite a bit more flexibility in how sounds are processed. Conceptually it is composed of a tree of audio nodes. Some of those nodes may be sources which just emit audio data, some of them may take in one or more inputs and output an altered stream, or they may be sinks which just consume the audio data.

In this demo I’m going to load the same clip in to an ArrayBuffer and play it. WebAudio can use an audio element as a source, but I’ll show that below in the section on playing music.

Demo

<button id="launch" onclick="launch()">Launch</button>
<button id="play" onclick="play()" disabled>Play</button>

let audioContext;
let clipBuffer;

async function launch() {
    audioContext = new AudioContext();

    let response = await fetch('soundeffect.wav');
    let clipBytes = await response.arrayBuffer();
    clipBuffer = await audioContext.decodeAudioData(clipBytes);

    document.getElementById('launch').setAttribute('disabled', '');
    document.getElementById('play').removeAttribute('disabled');
};

function play() {
    let clipSource = new AudioBufferSourceNode(audioContext, { buffer: clipBuffer });
    clipSource.connect(audioContext.destination);
    clipSource.start();
}

Here there is no audio element required. What is required though is that the core audio object, AudioContext, is created in response to a user gesture. This is the same constraint that’s on requestFullscreen and requestPointerLock, and is why there is now a launch button. In an eventual game this could be done in the handler for the launch game button.

Once the clip is downloaded in to an ArrayBuffer, it’s just a case of calling decodeAudioData to process it in to a playable format.

Also, I’m creating a new AudioBufferSourceNode each time play is clicked. This node is then connected directly to the audio output node, audioContext.destination, and started. As a result I can click play as fast as I want and each click will trigger the clip even if it results in overlapping instances of it being played.

Play Music

There are some important differences between playing a single clip and a playing music. The most notable is that music tends to be looped. Another is that there usually isn’t the same kind of real-time constraints on music - it may not be game breaking for music to take a few frames to start playing. I’m also more likely to want to stop and start music, where sfx clips are more likely to be fire-and-forget.

Audio Element

Again, I’m starting with an example of playing music using just the audio element. This is quite similar to the clip example:

Demo

<audio id="music" loop="true"></audio>

let musicElement;

window.addEventListener('DOMContentLoaded', (event) => {
    musicElement = document.getElementById('music')

    musicElement.addEventListener('canplay', () => {
        document.getElementById('play').removeAttribute('disabled');
        document.getElementById('stop').removeAttribute('disabled');
    })

    musicElement.src = 'music.wav';
});

function play() {
    musicElement.play();
}

function stop() {
    musicElement.pause();
    musicElement.currentTime = 0;
}

The audio element gets its loop attribute set, which will make the music loop once playing. I’m also subscribing to the canplay event. This will fire when enough of the file has been downloaded that play can start. Since music files can be large but should download faster than they can play this would let me start music playback without having to wait for the full track to complete downloading.

The pause method on the audio element doesn’t seek back to the beginning of the track, so my stop function also resets the time back to the beginning of the track.

WebAudio

I can also fetch the full music track, and play it similarly to the WebAudio clip example:

Demo

let audioContext;
let musicBuffer;
let musicSource;

async function launch() {
    audioContext = new AudioContext();

    let response = await fetch(url);
    let clipBytes = await response.arrayBuffer();
    musicBuffer = await audioContext.decodeAudioData(clipBytes);

    document.getElementById('launch').setAttribute('disabled', '');
    document.getElementById('play').removeAttribute('disabled');
    document.getElementById('stop').removeAttribute('disabled');
};

function play() {
    if (musicSource) return;

    musicSource = new AudioBufferSourceNode(audioContext, { buffer: musicBuffer, loop: true });
    musicSource.connect(audioContext.destination);
    musicSource.start();
}

function stop() {
    musicSource.stop();
    musicSource = null;
}

The main difference here is the addition of the loop: true option when I create the AudioBufferSourceNode from the AudioBuffer. I also guard the play function to prevent the music being played multiple times as once.

This approach works perfectly well. The main downside is having to fetch and decode the entire music file before playback can begin. This could be a bit wasteful for a music track. The upside is that music playback will happen exactly in line with the AudioContext timer, which would be required for any kind of beat matching game.

WebAudio with Audio Element

I can also get the best of both worlds - getting the power of the WebAudio framework while still being able to streaming the music to the audio element:

Demo

<audio id="music" loop="true"></audio>

let audioContext;
let musicElement;
let musicSource;

function launch() {
    musicElement = document.getElementById('music');

    musicElement.addEventListener('canplay', () => {
        document.getElementById('play').removeAttribute('disabled');
        document.getElementById('stop').removeAttribute('disabled');
    })

    musicElement.src = 'music.wav';

    audioContext = new AudioContext();
    musicSource = audioContext.createMediaElementSource(musicElement);
    musicSource.connect(audioContext.destination);
}

function play() {
    musicElement.play();
}

function stop() {
    musicElement.pause();
    musicElement.currentTime = 0;
}

This is very much a mix of the previous examples. There is an audio element that is controlled the same way as before, and which fires the same events. And there is an AudioContext providing an audio node graph. The glue is the createMediaElementSource which creates an audio node which has a single output containing the sound being played by the audio element.

Volume Control

So far there hasn’t been a particularly compelling reason to use WebAudio over a collection of audio elements. This is where I’m going to start using the audio nodes to manipulate playback.

In this first example I’m going to add a single GainNode. This node will control the volume of the clip. The source node will connect to the gain node instead of the AudioContext.destination and the gain node will be the node connected to the destination instead:

Demo

<input type="range" id="volume" name="volume" 
       min="0" max="100" value="100"
       onchange="setVolume(event)" disabled>

let audioContext;
let clipBuffer;
let gainNode;

async function launch() {
    audioContext = new AudioContext();

    gainNode = audioContext.createGain();
    gainNode.connect(audioContext.destination);

    let response = await fetch('soundeffect.wav');
    let clipBytes = await response.arrayBuffer();
    clipBuffer = await audioContext.decodeAudioData(clipBytes);

    document.getElementById('launch').setAttribute('disabled', '');
    document.getElementById('play').removeAttribute('disabled');
    document.getElementById('volume').removeAttribute('disabled');
}

function play() {
    let clipSource = new AudioBufferSourceNode(audioContext, { buffer: clipBuffer });
    clipSource.connect(gainNode);
    clipSource.start();
}

function setVolume(event) {
    let value = parseInt(event.target.value, 10) / 100;
    gainNode.gain.setValueAtTime(value, audioContext.currentTime);
}

The launch function creates the new gain node and connects it to AudioContext.destination. play now connects the source node to the gain node.

The new setVolume function adjusts the value of the gain node using setValueAtTime. It is possible to directly set the gain parameter value using gainNode.gain.value = X, but this could result in some surprising interactions with the parameter ramp functions I’ll be covering in the next section.

And now to further demonstrate the flexibility of the audio node graph, the following demo plays both short clips and music and presents the traditional set of audio sliders - master, music and sound effect. It’s using an audio element for the music, and an AudioBuffer source for the sound clip, it bring together a lot of what was covered so far:

Demo

function launch() {
    audioContext = new AudioContext();

    masterGainNode = audioContext.createGain();
    masterGainNode.connect(audioContext.destination);

    musicGainNode = audioContext.createGain();
    musicGainNode.connect(masterGainNode);

    effectGainNode = audioContext.createGain();
    effectGainNode.connect(masterGainNode);

    // [.. Load soundeffect.wav clip ..]

    musicElement = document.getElementById('music')
    // [.. Load music.wav ..]

    musicSource = audioContext.createMediaElementSource(musicElement);
    musicSource.connect(musicGainNode);
}

function playEffect() {
    let clipSource = new AudioBufferSourceNode(audioContext, { buffer: effectBuffer });
    clipSource.connect(effectGainNode);
    clipSource.start();
}

Here I’m creating three gain nodes. One for sound effects. One for music. And one for master volume which both the previous nodes feed in to. Diagramming what the graph looks like:

        +---------------+                 +----------------+
        |   <audio>     |                 |   AudioBuffer  |
        +---------------+                 +----------------+
                |                                |
+-----------------------------+       +------------------------+-+-+
| MediaElementAudioSourceNode |       |  AudioBufferSourceNode | | |
+-----------------------------+       +------------------------+-+-+
                |                                |
        +---------------+                 +----------------+
        | musicGainNode |                 | effectGainNode |
        +---------------+                 +----------------+
                |                                |
                +----------------+---------------+
                                 |
                         +----------------+
                         | masterGainNode |
                         +----------------+
                                 |
                         +-------------+
                         | destination |
                         +-------------+

Fades

A nice-to-have that I always end up implementing early is smoothly fading out clips when I need to stop them mid way through. On the previous music demos, when I hit stop there is an audible crackle. This is caused by the sound wave going immediately from some amplitude to zero, creating the edge of a square wave.

To fix this, when stopping the music a quick fade to zero volume before stopping the track will prevent that hard edge causing a crackle. Fortunately the AudioParams of the audio nodes support automatic transitioning to a new value over a given duration. I’m going to use exponentialRampToValueAtTime in this case:

Demo

function hardStop() {
    musicElement.pause();
    musicElement.currentTime = 0;
}

function softStop() {
    musicGain.gain.setValueAtTime(musicGain.gain.value, audioContext.currentTime);
    musicGain.gain.exponentialRampToValueAtTime(0.0001, audioContext.currentTime + 0.2);
    setTimeout(() => {
        musicElement.pause();
        musicElement.currentTime = 0;
    }, 200);
}

The difference between pressing the hard stop and soft stop buttons in the demo show pretty well the effect to the fade.

An important note is that exponentialRampToValueAtTime ramps to the destination value from the previously set scheduled value, over the period from when that previous value was set to the ramps destination time. That’s why I first have to setValueAtTime so that the previous scheduled value is the current value at the current time. Once the transition is finished the audio element is stopped.