Sonification in Web Based Applications using JavaScript

Generally speaking, there is an infinite amount of possibilities to create and trigger sound on a computer, such as using combinations of MIDI and/or pureData, Processing, OpenFrameworks, software synthesizers, Java/JavaScript or C++/C#.

When developing sonification for a web based environment, several options using HTML5 and/or JavaScript libraries exist, to create sounds or play sound files. Some of these possibilities in relation to classic sonification techniques have been examined closer. An explanation of sonification techniques can be found in a previous blog post here.

HTML5

There are different possibilities to embed audio in a website without using JavaScript. Using the <audio controls> tag introduced in HTML5, audio files can be integrated into a website the following way:

<audio id="audiotag1" src="audio/flute_c_long_01.wav" preload="auto"></audio>

Using simple JavaScript methods, these sound files can then interactively be triggered:

<audio id="audiotag1" src="audio/flute_c_long_01.wav" preload="auto"></audio>
<a href="javascript:play_single_sound();">Play 5-sec sound on single channel</a>
<script type="text/javascript">
    function play_single_sound() {
        document.getElementById('audiotag1').play();
    }
</script>

This is a good and effective way to play sound in browsers, though older browsers are not supported. Furthermore, there are no dynamic effects available, so all alterations to the sound have to be happen somewhere in the back end. Therefore only auditory icons could be triggered this way, any sonification techniques that need sounds to dynamically change with the data (such as earcons or parameter mapping) won't be easily implemented.

howler.js

 

Howler.js is a JavaScript library used to trigger audio files, such as *.mp3, *.ogg or *.wav. Parameters that can be altered are volume and panning, which is even possible in surround 3D. Additionally, fade in and out effects are implemented as well. A loop flag can be set to constantly repeat a sound, so continuous sound waves are theoretically possible as well. 

A simple sound is defined like the following:

var sound = new Howl({
  urls: ['sound.mp3']
}).play();

Setting the loop flag and altering the volume is done in the following way:

var sound = new Howl({
  urls: ['sound.mp3', 'sound.ogg', 'sound.wav'],
  autoplay: true,
  loop: true,
  volume: 0.5,
  onend: function() {
    console.log('Finished!');
  }
});

Disadvantages using this particular library are the absence of audio filters or any other audio effects.

Therefore, this library won't be useful to play earcons, as no parameters can be altered to change the sound. For classic auditory icons however, this library proves to be extremely handy.

jsfx.js

 

jsfx.js is roughly speaking an online synthesizer, that can create dynamic synthesized sounds. There are various parameters that can be altered dynamically, such as the ADSR curve (Attack, Decay Sustain, Release), Slide, Vibrato, Phaser or LP and HP Filters. A new sound is defined the following way:

audioLibParams = {
    test : ["noise",0.0000,0.4000,0.0000,0.0060,0.0000,0.1220,20.0000,460.0000,2400.0000,-0.5240,0.0000,0.0000,0.0100,0.0003,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.9990,0.0000,0.0000,0.0000,0.0000]
  };

  samples = jsfxlib.createWaves(audioLibParams);
  samples.test.play();
  samples.explosion.play();

This array contains all information about the sound being triggered and can be created with an online tool.  

To alter those values interactively, like mapping data values to particular effects, one has to know which value represents which parameter. As all these values have completely different ranges and represent entirely different sound parameters, using this library will prove difficult to achieve the desired outcome in the sound design. Another disadvantage that flagged during first experiments was, that problems occurred when trying to trigger multiple sounds in parallel. What is also missing is a way to generate a continuous sound. As all sounds have an ADSR curve, a continuous wave generator is not implemented. 

Therefore, this library won't be useful for the parameter mapping sonification technique, as it is not possible to create continous sounds. For auditory icons and espicailly earcons however, this library contains high potential and is very suitable, as all triggered sounds can be altered and a large number of paramters can be changed every time before the sound is triggered. Experimenting with the online tool to create new sounds points out to be very fun and exciting.

timbre.js

timbre.js is sound library to create and influence sound waves very similar to a standard synthesizer. The library is well documented and appears to be very powerful. It also possible to trigger audio files. A simple sine wave is already initialized the following way:

T("sin", {freq:880}).play();

Several wave forms can be created, as well as noise.
Various effects, such as tremolo, vibrato, filtering or phaser effects can be implemented: Since it is possible to code very deep with this library and all the sound synthesis is generated with raw numbers, it is possible to create any desired sound effect:

var freq = T("pulse", {freq:5, add:880, mul:20}).kr();

T("sin", {freq:freq, mul:0.5}).play();

To create beautiful and meaningful sound design with this library however appears to be very time consuming, as most things have to be coded by hand (compared to jsfx.js, which already comes with a ready synthesizer interface). The possibilities using this sound library on the other hand are immense and it appears to be one of the most powerful sound libraries for JavaScript out there.

First experiments with each of those sound libraries in combination with data being read from a csv file using the data visualization library d3.js can found here.

References

GitHub. 2013. timbre.js. [online] Available at: https://github.com/mohayonao/timbre.js/ [Accessed: 25 Sep 2013].
GitHub. 2012. jsfx. [online] Available at: https://github.com/egonelbre/jsfx [Accessed: 25 Sep 2013].
GitHub. 2013. howler.js. [online] Available at: https://github.com/goldfire/howler.js [Accessed: 25 Sep 2013].
W3schools.com. 2013. HTML5 Audio. [online] Available at: http://www.w3schools.com/html/html5_audio.asp [Accessed: 25 Sep 2013].