Learn Web Audio from the Ground Up, Part 4:

Additive Synthesis And the Harmonic Series

Sine waves are simple things and provide a nice entry point to learning audio signal processing. So far we've been able to discuss things like frequency and amplitude using nothing but individual sine wave oscillators.

But the thing is, a solitary sine wave is not very interesting to listen to. It's a blank slate, devoid of any color, character, or drama.

Things start to get much more interesting when we have multiple sine wave oscillators at our disposal. What we can do is combine them to produce more complex sound waves. This technique is called additive synthesis.

What Happens When I Combine Two Or More Sine Waves?

As we discussed in Part 1, a digital sound signal is essentially just a (very large) array of individual numbers that represent the sound's amplitude over time.

When we want to combine two sound signals together, all we have to do is add the samples from both signals together for each time step.

// Pseudocode
for (let i = 0 ; i < signal1.length ; i++) {
  combo[i] = signal1[i] + signal2[i];
}

What we end up with when we do this is a third sound signal that represents the combination of the originals. For two sine wave inputs, we get something that no longer looks like a sine wave:

We have just synthesized a wave using additive synthesis.

When we combine signals like this, some samples end up having a higher amplitude than in either of the originals, which makes sense because what we're doing is adding signals together. But some samples are lower than either of the originals, which is because sine waves also oscillate to the negative side of the x-axis, and we end up effectively subtracting samples.

The extreme form of this kind of subtraction is with two waves that are exactly opposite of each other: Waves that have the same frequency and amplitude, but are in opposite phases. They completely cancel each other out, leaving nothing but silence.

This is what happens in noise-canceling headphones. They synthesize sound waves in real time that are the exact opposite of the environmental noise reaching your ears. The goal is that you end up hearing none of that noise because the sounds cancel each other out.

When we want to do additive synthesis in Web Audio, we luckily don't actually need to sum up any numbers manually. The API allows connecting two or more source nodes to the same destination. When this happens, it does the addition for every sample automatically behind the scenes. So for example, we can spin up three OscillatorNodes, start them at the same time, and connect them to the destination. What we hear is the combination wave synthesized from those three oscillators.

Usually we also want to have at least one gain involved though. That's because when we are adding individual signals that are in the [-1, 1] range, we very quickly get combined signals that don't fit in that range, and we have seen how that may cause clipping to occur. We can use a gain to multiply the combined signal with some constant number (like 0.5) to scale it back down.

Here's an example that uses three oscillators to make an approximate major triad chord above the A4 note, and a gain to scale it down.

let audioCtx = new AudioContext(); Run / Edit

let osc1 = audioCtx.createOscillator();
let osc2 = audioCtx.createOscillator();
let osc3 = audioCtx.createOscillator();

let masterGain = audioCtx.createGain();

osc1.frequency.value = 440;
osc2.frequency.value = 550;
osc3.frequency.value = 660;
masterGain.gain.value = 0.3;

osc1.connect(masterGain);
osc2.connect(masterGain);
osc3.connect(masterGain);
masterGain.connect(audioCtx.destination);

osc1.start();
osc2.start();
osc3.start();

What Is The Harmonic Series?

We just combined three sine waves with three different frequencies to produce a musical major triad. Of course, this is just one of an infinite amount of frequency combinations we could have used. How can we navigate this infinite frequency space to find interesting sounds?

Well, one way to do it is to pick one frequency - any frequency - and then produce integer multiples of that frequency. For example, starting from 440Hz we get the following series:

• 440Hz
• 2 * 440Hz = 880Hz
• 3 * 440Hz = 1320Hz
• 4 * 440Hz = 1760Hz
• 5 * 440Hz = 2200Hz
• 6 * 440Hz = 2640Hz
• ...

This is called a harmonic series. We can mount such a series on any fundamental frequency (like 440Hz). Then we add integer multiples to it, called harmonics or overtones (880Hz, 1320Hz...).

The reason why this relationship between frequencies is special is that it occurs naturally in the physical world. For example, when we pluck the string of a guitar or a violin, it vibrates on a certain frequency determined by the string length (L), but at the same time it also vibrates also on a number of higher frequencies, which happen to be based on integer divisions of the original string length: L/2, L/3, L/4...

So when you pluck a string instrument, you're not generating a sine wave or any purely single-frequency sound. You're producing a combination of sounds across these different frequencies – a harmonic overtone series. The same is true for all natural sounds out there in the world.

Now, the relationships between frequencies in the harmonic series are not only natural, but also inherently musical, and that is to do with how they produce musical intervals.

As we have discussed, there's an exponential relationship between frequencies and musical pitches. When we get higher on the frequency spectrum, the same absolute distance in frequency spans a much smaller space in musical notes than on the lower ends. Since the distance between every two frequencies in a harmonic series is constant (equal to the fundamental frequency), the musical intervals between consecutive harmonics get smaller and smaller the higher we get.

What every harmonic series in fact includes at its base is a set of diminishing musical intervals:

These are the most fundamental intervals in western music. They were famously used by Richard Strauss in the opening "Sunrise" fanfare of "Also Sprach Zarathustra". The intervals between the first four notes are a perfect fifth, a perfect fourth, and a major third.

How Can I Synthesize Sounds Using The Harmonic Series?

We can use the above technique of combining sine waves to put together a number of harmonics from the harmonic series. Here's a combination of four waves - a fundamental and three of its harmonics from the series that we just visualized:

What's really interesting about this is is that, unlike with the major triad we made earlier, this time you don't actually hear three different sounds. You only hear the lowest one, A4, but it no longer sounds like just a sine wave!

What's happening in your brain is something called "fusion". Your auditory system combines harmonically related signals so that it sounds like all you're hearing is the fundamental frequency. The harmonics only serve to add "color" to the sound, changing its timbre.

This is the key idea behind additive synthesis. By combining different harmonics (as well as non-harmonic frequencies) and varying their relationships over time, we can produce different kinds of sounds.

Here's how we can produce this particular timbre in Web Audio:

let audioCtx = new AudioContext(); Run / Edit

let fundamental = audioCtx.createOscillator();
let overtone1 = audioCtx.createOscillator();
let overtone2 = audioCtx.createOscillator();
let overtone3 = audioCtx.createOscillator();

let overtone1Gain = audioCtx.createGain();
let overtone2Gain = audioCtx.createGain();
let overtone3Gain = audioCtx.createGain();
let masterGain = audioCtx.createGain();

fundamental.frequency.value = 440;
overtone1.frequency.value = 880;
overtone2.frequency.value = 1320;
overtone3.frequency.value = 1760;

overtone1Gain.gain.value = 0.1;
overtone2Gain.gain.value = 0.2;
overtone3Gain.gain.value = 0.5
masterGain.gain.value = 0.3;

fundamental.connect(masterGain);
overtone1.connect(overtone1Gain);
overtone2.connect(overtone2Gain);
overtone3.connect(overtone3Gain);
overtone1Gain.connect(masterGain);
overtone2Gain.connect(masterGain);
overtone3Gain.connect(masterGain);
masterGain.connect(audioCtx.destination);

fundamental.start(0);
overtone1.start(0);
overtone2.start(0);
overtone3.start(0);

In addition to the oscillators we saw earlier, this time we're also using individual gains, one for each frequency. We want to mix in different harmonic partials with different amplitude relationships to get the sound we want. In this case, we're using the full amplitude for the fundamental, and then 0.1, 0.2, and 0.5 for the harmonics, respectively.

We can package this all up into an ES2015 class so that we have a "synth" we can use to play any musical notes, similarly to what we did using plain sine waves earlier. We can play notes on any frequencies, and as long as the relationship between the partials stay the same, we'll maintain the same timbre for our "instrument":

let audioCtx = new AudioContext(); Run / Edit

class HarmonicSynth {

  /**
   * Given an array of overtone amplitudes, construct an additive
   * synth for that overtone structure
   */
  constructor(partialAmplitudes) {
    this.partials = partialAmplitudes
      .map(() => audioCtx.createOscillator());
    this.partialGains = partialAmplitudes
      .map(() => audioCtx.createGain());
    this.masterGain = audioCtx.createGain();

    partialAmplitudes.forEach((amp, index) => {
      this.partialGains[index].gain.value = amp;
      this.partials[index].connect(this.partialGains[index]);
      this.partialGains[index].connect(this.masterGain);
    });
    this.masterGain.gain.value = 1 / partialAmplitudes.length;

  }

  connect(dest) {
    this.masterGain.connect(dest);
  }

  disconnect() {
    this.masterGain.disconnect();
  }

  start(time = 0) {
    this.partials.forEach(o => o.start(time));
  }

  stop(time = 0) {
    this.partials.forEach(o => o.stop(time));
  }

  setFrequencyAtTime(frequency, time) {
    this.partials.forEach((o, index) => {
      o.frequency.setValueAtTime(frequency * (index + 1), time);
    });
  }

  exponentialRampToFrequencyAtTime(frequency, time) {
    this.partials.forEach((o, index) => {
      o.frequency.exponentialRampToValueAtTime(frequency * (index + 1), time);
    });
  }
}


const G4 = 440 * Math.pow(2, -2/12);
const A4 = 440;
const F4 = 440 * Math.pow(2, -4/12);
const F3 = 440 * Math.pow(2, -16/12);
const C4 = 440 * Math.pow(2, -9/12);

let synth = new HarmonicSynth([1, 0.1, 0.2, 0.5]);

let t = audioCtx.currentTime;
synth.setFrequencyAtTime(G4, t);
synth.setFrequencyAtTime(G4, t + 0.95);
synth.exponentialRampToFrequencyAtTime(A4, t + 1);
synth.setFrequencyAtTime(A4, t + 1.95);
synth.exponentialRampToFrequencyAtTime(F4, t + 2);
synth.setFrequencyAtTime(F4, t + 2.95);
synth.exponentialRampToFrequencyAtTime(F3, t + 3);
synth.setFrequencyAtTime(F3, t + 3.95);
synth.exponentialRampToFrequencyAtTime(C4, t + 4);

synth.connect(audioCtx.destination);
synth.start();
synth.stop(audioCtx.currentTime + 6);

There's a vast space of timbres one can generate from the harmonic series, and you can now easily explore it by constructing HarmonicSynths with different overtone structures. Here are a couple of more interesting examples.

We can produce a sawtooth wave () by oscillating on every frequency of the harmonic series, with amplitudes that decrease as the frequencies increase: The first harmonic has an amplitude of 1/2 of the fundamental, the second has 1/3 and so on:

let partials = []; Run / Edit
for (let i=1 ; i <= 45 ; i++) {
  partials.push(1 / i);
}
let synth = new HarmonicSynth(partials);

synth.setFrequencyAtTime(440, audioCtx.currentTime);
synth.connect(audioCtx.destination);
synth.start(0);

Here, we are using a bank of 45 oscillators. In theory, there's an infinite amount of harmonics in the harmonic series and you would need all of them to make a perfect sawtooth shape. But when making sounds we only need to go up to the upper reaches of human hearing, which, as we discussed earlier, is no more than 20kHZ.

If we go and eliminate all the even harmonics from our sawtooth wave, we end up with a square wave () with yet another kind of timbre:

let partials = []; Run / Edit
for (let i=1 ; i <= 45 ; i++) {
  if (i % 2 !== 0) {
    partials.push(1 / i);
  }
}
let synth = new HarmonicSynth(partials);

synth.setFrequencyAtTime(440, audioCtx.currentTime);
synth.connect(audioCtx.destination);
synth.start(0);

I've made a little interactive application you can use to further explore the harmonic series, or at least the first 12 harmonics in it:

Beating

We've barely scratched the surface of what we can do with additive synthesis, and to conclude we're going to discuss one more interesting technique, which is to combine sound waves that have almost but not quite the same frequency. The interference pattern between the waves changes gradually over time as they fall in and out of phase:

This effect is called beating. It is a result of the gradually changing wave phases shifting between reinforcing each other and canceling each other. The resulting effect is pretty remarkable, and we can hear it by setting up oscillators that have nearly the same frequency. For example, using frequencies 330 and 330.2 we get a sound that comes and goes every few seconds:

let audioCtx = new AudioContext(); Run / Edit

let osc1 = audioCtx.createOscillator();
let osc2 = audioCtx.createOscillator();
let gain = audioCtx.createGain();

osc1.frequency.value = 330;
osc2.frequency.value = 330.2;
gain.gain.value = 0.5;

osc1.connect(gain);
osc2.connect(gain);
gain.connect(audioCtx.destination);

osc1.start();
osc2.start();
osc1.stop(audioCtx.currentTime + 20);
osc2.stop(audioCtx.currentTime + 20);

By combining several of these beating pairs together, we get more and more interesting patterns: Different beating intervals make different pairs come and go at different speeds, resulting in seemingly complex changes over time. There are shifts not only in timbre, but also in the perceived pitch of the sound:

let audioCtx = new AudioContext();  Run / Edit

let osc1 = audioCtx.createOscillator();
let osc2 = audioCtx.createOscillator();
let gain1 = audioCtx.createGain();

let osc3 = audioCtx.createOscillator();
let osc4 = audioCtx.createOscillator();
let gain2 = audioCtx.createGain();

let osc5 = audioCtx.createOscillator();
let osc6 = audioCtx.createOscillator();
let gain3 = audioCtx.createGain();

let masterGain = audioCtx.createGain();

osc1.frequency.value = 330;
osc2.frequency.value = 330.2;
gain1.gain.value = 0.5;

osc3.frequency.value = 440;
osc4.frequency.value = 440.33;
gain2.gain.value = 0.5;

osc5.frequency.value = 587;
osc6.frequency.value = 587.25;
gain3.gain.value = 0.5;

masterGain.gain.value = 0.5;

osc1.connect(gain1);
osc2.connect(gain1);
gain1.connect(masterGain);

osc3.connect(gain2);
osc4.connect(gain2);
gain2.connect(masterGain);

osc5.connect(gain3);
osc6.connect(gain3);
gain3.connect(masterGain);

masterGain.connect(audioCtx.destination);

osc1.start();
osc2.start();
osc3.start();
osc4.start();
osc5.start();
osc6.start();