Categories
dynamic sound generation popforge

audio/video sequencer

Pipslab is a very successful collective of multimedia artists and through Tim Hudson of Flashclub they have asked me to port a kind of audio/video sequencer from Director to Flash. Though Director has a rather steady internal timer, on older processors you’ll notice the same kind of sluggishness as the ENTER_FRAME event in Flash. So the challenge was to create a steady clock in Flash to sync both audio and video to.

I started by looking into the SOUND_COMPLETE event. When using a loop of 2048 or less samples, the exactness of this event varies roughly between 25 and 100 ms whereas it should be a steady 46 ms (1000/44100 * 2048 ≈ 46). So i used an old trick by adding some ms of silence in front of the actual sound. Whenever a SOUND_COMPLETE occurs i calculate the deviation compared to theoretical (multiple of) 46 ms and i compensate this by starting the sound earlier or later dependent on the offset:

sound.play(offset,1);

Although this works pretty well in the standalone player, in a browser on slower processors the beat is very unstable and i experienced hiccups and fall outs. So i started studying the Popforge library renowned for its steady beats.

I took me quite some time to grasp how it actually works but i became gradually clear to me after i had made a visualization of the principle. Below you can create a pattern and by clicking on start the sequencer starts.

Next, the red bars that appear demark the audio buffers and the green bars set apart the beats. If you increase the tempo you’ll notice that more beats fit in a single buffer.

As you can see, sometimes a single sample is split up and spread over 2 buffers. Because the buffers are seamlessly glued after each other you don’t hear the joints.

The indicator on top of the pattern-edit grid is not very precise, and if you increase the tempo it gets even lesser accurate. André Michelle made an online version of the Roland TR909 and if you look closely at this implementation of Popforge, you’ll notice that the leds above the pattern buttons aren’t very exactly either. But as the emphasis of the Popforge library lies on sound, this is perfectly valid.

However, in my case i had to deal with a video aspect as well. After some fruitless experiments i came up with a rather quick and dirty trick: i adapted the tempo as one beat to match exactly with the length of a buffer.

Downside of course is that i am tied to a very limited set of tempos. Buffersize 8192 (4 times 2048) boils down to 1000/(8192*1000/44100) * 60/4 ≈ 81 bpm. In the example above i use a buffer of 12288 bytes (6 times 2048) so a tempo of approx 54 bpm will align one beat to one buffer.

Anyway the result feels rather synced and steady and it performs way better than the version in Director. I am still left with an unanswered question though: the engine that glues the subsequent buffers after each other is the SOUND_COMPLETE event that is fired by a dedicated sync sound.

This sync sound is just an array of ‘silent bytes’ with a length that matches the buffersize (minus one). As soon as the application is initialized the sync sound starts silently playing. Whenever the SOUND_COMPLETE event arises the AudioBuffer class checks if any sample data is prepared in the buffer. If so, the data is played back and you actually hear something. In any case the sync sounds starts playing again, resulting in an endless sync stream.

Because the SOUND_COMPLETE event isn’t very steady, even at higher framerates (80 – 120 fps), i would expect to hear gaps and pops because a deviation of say 20 ms of the theoretical interval between 2 subsequent SOUND_COMPLETE events should be audible, and this deviation is not compensated somehow or other in the Popforge library.

In theory the first method i tried should give the same result as the Popforge library, if even better, because of the compensation of the deviation (the offset, see above), but it doesn’t.

However, if i test the swf above on a rather old processor (AMD Turion 64 ML28 1,6 GHz), Popforge starts stammering at and above 66 bpm, whereas the FL909 works like a charm at any tempo.

Since drawing the waveform is rather CPU intensive, i guess it causes the ENTER_FRAME event to become such instable that the SOUND_COMPLETE event can’t be taken care of in time, resulting in gaps between the sample slices. When using a buffer of 12288 bytes, the framerate must be at least 1000/(12288 * 1000/44100) ≈ 3,58, say 4 fps, and apparently with heavy drawing duty, the flashplayer sometimes drops below.

I have tried to compensate this with the offset trick described above, but until now without success. Still busting my brains on this.

2 replies on “audio/video sequencer”

Nice article.

Yeah, Adobe must obey the browsers, Firefox only allows 30 frames/sec or so. The Timer or EnterFrame events only try to do their best to give that tick at the desired framerate or miliseconds, but they only do their best, it will never be constant.

Sound is easier becuase the flash plugin only has to feed some buffers and the audio hardware will play them. Audio is the only thing that can be synced precisely in Flash.

I did tests with an Air application, wich give is more precise and can go faster, I wrote a post about that tetst.

http://wiibart.posterous.com/flash-framerate-in-a-air-application

I like your sequencer, did you use SampleDataEvent? (the event that asks for filling the buffers)

Bart
Bart

Leave a Reply to admin Cancel reply

Your email address will not be published. Required fields are marked *