Saturday, 23 May 2015

Synths as a service

Here is a snapshot (from a video of a Keynote presentation, so it is a bit grainy) of the state of play of the synths project, with a bit of under-the-hood on how it all straps together.

OK, from the top - the synth application contains 3 different types of synth. These are -

Synth 1 : a Virtual Analog synth 'PIANA', a monosynth with 2 full-fat oscillators OSCA and OSCB plus a sub-oscillator OSCC. OSCA and OSCB support limited phase distortion - effectively PWM of all waveshapes, compressing the first half of each cycle and expanding the second to leave the period and the fundamental unchanged. OSCA and OSCB also support continuous morphing of the waveshape between 2 endpoints - e.g. square to sin, saw to square - under modulation matrix control. OSCA and OSCB may be voiced independently, with a range from 32' to 4'. OSCC is always a sin, and is always voiced at 32', so if A and B are both 32' it just reinforces them, if they are both 8' it sounds 2 octaves below them. Optionally OSCC may be used instead as a white noise source if sub-bass isn't needed or if you are making analog drums. Or explosions, or jet fighters, whatever. OSCA OSCB and OSCC levels are under control of the modulation matrix, so different EGs can apply to them, allowing for nice 'ensemble bass' sounds when OSCC is set to a slightly slower attack than OSCA and OSCB. OSCC frequency is hard-locked to OSCA, divided down by any difference in feet, but OSCA and OSCB frequency are under control of the mod matrix, which lets me do that great 'Virginia Plain' guitar sound of an octave sweep up on every note. Which is obviously honoured by OSCC as it is tied to OSCA. Hard sync of OSCB to OSCA is supported - nasty!! - and there is a ring modulator for programmable amounts of AxB - doubleplus nasty!!! The filter is selectable between a 'Moogy' filter (4th order resonant LPF) and a 'Minibruty' filter (a 'Hal Chamberlin' state-variable filter with taps for LP, BP, HP, notch and peaking). The frequency responses of the filter models have been carefully tuned against each other so both can be used to 'play the filter' or to just allow resonance to act as a self-oscillating extra sound source. There are 4 EGs and 4 LFOs feeding into the mod matrix. Modulation is rich and frankly a bit crackers, but it gives it a semi-modular amount of flexibility. The oscillators are all deep down in their bones just sin wave generators, and antialiasing of the oscillators is achieved by waveshaping, backing off from harmonic generation towards a pure tone when there is a danger of harmonics being generated too close to, or beyond Nyquist. It sounds great, even if I say so myself, yet manages to burn relatively low amounts of CPU, allowing multiple instances on a first generation Raspberry Pi or similar embedded platform.

Synth 2 : a sample replay polysynth 'PIANATRON' which is designed to be as lightweight and fast as possible. Two layers of samples are supported per note, with up to 12 note polyphony per polysynth. The synth honours the pitch bend controller, and allows Mod Wheel to be mapped to the layer blend control, and allows layer B to be either 'cross-fade' (removing A as B is ramped up) or 'mix' (simply adding in B). There is an EG available for overall level (to allow long decays on looped samples for example) and layer B may be detuned or uptuned by an octave relative to layer A. No filters, no modulation. There is no antilaliasing whatsoever, so if you play notes stretched to more than about a fifth above the sampled frequency, expect degradation and some energy to break on through from the other side. Dead simple, dead fast, but with good samples it really sounds brilliant, even more so with some subtle detuning between layers.

Synth 2.5 : let's not forget 'PICUSSION', which is PIANATRON for drums. Exactly the same engine as PIANATRON, but restricted to just one layer per note, and with modified polyphony management.

and finally Synth 3 : a wavetable synth 'PICYCLE' which supports 2 layers of wavetable sound per note, up to 12 note polyphony per synth. There is an EG per layer, detuning / uptuning of layer B relative to layer A, and in addition 2 LFOs, one always on (the amount is programmed into the preset) and one dialled in by the mod wheel. I have typically used the 'always on' LFO for really low frequency modulation of one layer, to give movement on otherwise static tones. Each layer generates sound by passing through a 'trajectory' of up to 8 wavetables at programmable rate, different rates programmable per layer, and an LFO may be dialled in to influence the trajectory. The wavetables are lerped / morphed as the trajectory progresses, so the sound of even a very slow trajectory is a seamless sonic journey with no tonal discontinuities. There is a small number (less than 20) of wavetables hard-coded into the synth, but the cool thing for tweakers is that additional waves can be constructed using the built-in SIMD wavetable construction kit, documented here a few weeks back. A total of 128 waves is allowed in memory per run of the application, a number that maps onto the 'midi byte' selection capability of a MIDI program change.

The feature set may sound somewhat limited - limited modulation, no filters - but it is really a very good, rich and flexible synth engine, excellent for glacial, early 80s digital polysynth pads and beds. With resonant / formanty sounds in the tables (fundamental-less harmonics, windowed) it can sound surprisingly analog as well. Antialiasing of the oscillators is achieved by multi-octave representation and frequency-dependent run-time selection of the wavetables, akin to MIP-mapping in 3D graphics, and the results are very good indeed. 8 octaves of tables are stored for each voice, the tables are 1024 samples wide, so the lowest table represents a 43Hz fundamental, the highest 11kHz, which is an octave and a bit higher than the highest note on a piano - there is no danger of out of control aliases with this range of supported frequency.

A really great and surprisingly flexible polysynth, big sonic space, very nice sound quality, and one that I have not yet had time to dig into hard to fully explore its capabilities. Too busy writing and performance tuning it ...

And while I'm on the subject of performance tuning - on a Pi 1st generation (A+, B+, A, B) overclocked to 950MHz (my standard setting) I can manage just about 7-8 PIANAs with the wind behind me, plus my Open GL ES accelerated wiggly waveform display. I get about 3x as many PICYCLE notes (some 20-24 maximum PICYCLE notes sounding), and at least 2x that many PIANATRON notes. These are all somewhat fuzzy approximates because with the standard kernel in place, once the system approaches 75-80% load ALSA justs fall over occasionally and causes glitching. Bad glitching, not art glitching. It's gruesome. A real RT kernel would help, ALSA seems to spend a lot of time failing and insisting on EAGAIN. I do hate ALSA. A simple 'DirectDAC' abstraction would buy back at least 5% of the CPU and firm up the ground I'm standing on - I hope Gordon at IQaudIO is reading this - 'DirectDAC' ...

But anyway, I digress - with this bag of tools polyphony can be traded off against programmability and flexibility, by picking the right synth for the job. But you are not necessarily trading off any sonic richness - a carefully-sampled instrument will add vast richness at very low CPU cost, albeit with little sonic flexibility. So this synth collection offers a giant sonic palette to exploit.

The app allows up to 8 of each type of synth to be instantiated, for what would appear to be total of 24 available synths, but the real limit is set to 12 synths. This is partly due to MIDI channel count constraints, partly for performance reasons on the Pi. And partly because I wanted to keep some MIDI channels free for future cool stuff.

Above and beyond the synths are 4 stereo delay engines which are shared between the synths. Each synth can select to send into one of these 4. In addition a single reverb is strapped across the whole system, and not only does each synth have a 'reverb send' amount, each delay engine also has a 'reverb send', allowing really wet sounds but also placing the delayed sounds into the same reverberant space as the dry synth sound, glueing arrangements together nicely. The delays and reverb each implement a simple 6dB/octave LPF on their input, with programmable cutoff frequency. The reverb sports configurable reverb time, the delays a configurable delay time for the left and right path, feedback levels for left and right. This means it can do the Brian May 'Brighton Rock' thing and more due to the feedback. Check out this short video for an example of a single PIANA monosynth being manipulated live with reverb and delay send amounts being twiddled. Giant and fab, and it's just a single PIANA, driven by a sequence so that my hands are free for knobs - a first generation Pi can do 7 or 8 of these depending on filter and oscillator settings :

(eagle-eyed viewers of a nerdy disposition will realize, after a quick inspection of the waveforms, that this video predates the introduction of DC traps into the synths ... fascinating, Jim) 

There is a single inter-app / inter-thread MIDI port (accessible over ALSA using the virtual port name "SERVICE_MIDI") which allows a maximum of 16 different addressable ports and hence 16 different devices / synths to hang off it. The current setup allows MIDI channels 1 through 12 to have synths assigned to them - this is where the 12 synth limit comes in. Channel 16 is special, and 13 14 and 15 are currently reserved.

Channel 16 implements a configuration and plumbing engine, so the whole multi-synth system can be set up just using MIDI protocol. Messages sent down channel 16 do things like assign a synth of a given type to a given MIDI channel (e.g. 'activate wavetable synth number 2, set it up with 6 note polyphony, map it to MIDI channel 1, tie it to delay unit number 3'), and set up parameters for the reverb and the 4 delay engines. The sequencer allows all this to be programmed.

Once setup is done the synths are ready to accept normal MIDI commands down the newly wired-up channels to make music. As well as note on / note off / pitch bend / mod wheel the synths all respond to program change so that a keyboard with a program change button you be used to move between presets. All synths also respond to master volume, reverb send and effect send (controllers 91 and 92) for delay and reverb. The polyphonic synths respond to sustain pedal, and a bunch of other CCs are passed through for live tweaking. So with a MIDI controller that has enough knobs and buttons you can control everything from the keys. The application intercepts certain commands and chooses to act on them rather than route them - transport controls for example. In fact transport control are used to break out of 'repeat' statements in the sequencer. A sequencer program can set up a drum / bass break to last forever, allowing knob twiddling to mess with filters and oscillators, and hitting the 'play' transport control button breaks out of the repeat loop and allows the program to proceed. So if the crowd isn't digging the twiddling, break out of the loop early by hitting the '▶︎' button on the keyboard.

So, finally, onto the title of this post - 'Synths as a service'. Typically the app gets closed down and restarted every time I run it, which induces a pretty big startup delay as the big sample libraries get loaded into memory every time That delay can be even bigger if the demo is going to run a big MIDI capture file that not only needs loading and parsing before startup but also typically has a bit of captured silence at the start. So, by using the /service parameter and backgrounding the app with an & (yes, I should eventually do this internally and make it a daemon ... ) the app stays resident, listening to the ALSA virtual MIDI port and any connected USB MIDI devices, allowing 3rd party apps to instantaneously fire up and make music.

So looking to a future when this thing is done enough to get out there, if you have a MIDI binding available in your programming language that lets you see ALSA virtual MIDI ports, you will be able to directly code your own music with this engine.

Friday, 8 May 2015

DC traps introduced into all synths and reverb engine

Phase distortion / PWM biases horribly, so I just put a DC trap into each synth, and one into the reverb engine, to elminate any accumulated bias and buy back about 2x dynamic range for the system. And I just revamped all the delays and the architecture of the reverb engine to give a much more analog sound.

Now I need to spend a few day on iOS stuff, as 8.3 has apparently broken the operation of AudioBus on a bunch of apps out there. So I need to fix mine.

Thursday, 23 April 2015

And the Virtual Analog filter system just got a spring clean

It had got messy - lots of different filters bolted in as I experimented over time - now it's much sweeter.

Internally it now contains just two filters, the 'Moogy' 4th-order LP, plus a classic Hal Chamberlin state-variable filter. So you can dial up a 4th order LPF or any of the Chamberlin modes - effectively it's both a Moog and a Minibrute from a filter perspective.

There's been one tiny loss of niceness - the analog drums and the Glitterclap aren't quite as kicky / bitey as they were, but there's tweaking can be done yet, and it's a slight performance improvement over the old chained Butterworths.

UPDATE - I've grown to really like the sound of this Chamberlin filter over the last 2 days ... I'm enjoying having more high end leak through in the clap as a result of the shallower roll off. However, on filters in general, after a thorough implementation review of all these different filters and how I had strapped them in, I have realized there really is only one way to implement the innards of a synth in software. Every time, just ask yourself the question - 'What would Bob do?'  - and if the answer to that isn't obvious, do some research to work out how Bob did it. He got pretty much everything right in the core implementation of the Minimoog.

UPDATE UPDATE - and the final pass over the filters ... let each of them run for 10 minutes, resonating forever, calculating the frequency at which they are resonating, and map that frequency to the internal gain parameter within the filter. This is to capture an accurate tuning table for each filter, and requires a lot of excess work - 'computational oversampling' if you will - as the mapping is sort of backwards. In the filter's innards you set the gain value and as a result you get a filter with a certain resonant frequency. Whereas for a synth obviously you want to set the resonant frequency and have the synth magically set the right internal filter gain coefficient. I had done this a while ago for the 'Moogy' filter but its architecture just changed slightly and I wanted to be sure that the table was still valid given the changes, so took the opportunity to run both filters through the autotuner.

Now both have 'playable filters' (if anyone cares about that, it's so old school but is simply assumed in any digital synth) but importantly the two filters really are frequency matched throughout their range. Both respond identically to the same internal virtual control voltages, and I can now dial up two presets, one with the Moogy LP and one with a Chamberlin, and flick between them to hear the differences. And it really is just that bit more raucous and sizzly on the Chamberlin, a bit brighter as more HF leaks through due to the shallower filter. Nice to have them behaving correctly. And the tuning deliberately isn't totally precise (I lost the will to live waiting for an ultra-precise match to complete, so coarsened up the search grid and restarted), so there will be some 'analog' tuning issues of the filter's resonance vs. the oscillator's frequency, a few tens of cents' worth anyway.

Here is a video after the cleanup but before the final tune-up -

Friday, 17 April 2015

This wavetable synth is done

Time to call it a day and say yes, this thing is feature complete.

Feature set :

Single EG per note (only used for overall level)
2 LFOs per polysynth
- LFOs have sin, saw, square, tri and S/H modes
- LFOs may optionally be synced to sequencer clock - the one feature not *quite* implemented yet
- one LFO is always on, the other is ramped up by the Mod wheel

2 layers of wavetable trajectories ('oscillators') per note
Each wavetable trajectory consists of up to 8 wavetables, traversed in one of
- linear time
- quadratic time t*t
- inverse quadratic time 1-(1-t)*(1-t)
- exponential time - half-life a go-go
The layers are traversed independently, so A may be linear, B may be exponential for example

Without an EG to drive trajectory there is just a dTrajectory/dt parameter. Trajectory resolution is always normalized to 1.0 irrespective of how many tables are in the trajectory, so a dTrajectory/dt of 0.5 will take 2s to pass through all the tables, unless the linearity parameter is set to exponential, in which case the 0.5 is interpreted as a half-life, so after 0.5s you will be half way through the trajectory list, after 1s at 0.75 etc. I will probably do one minor change here and should reciprocate  this so the the exponential half life is half of the linear traversal duration. Exponential is nice in one key respect, you always have some movement going on, no matter how long you hold a note. But obviously you are getting asymptotic to 1.0 after a while.

The trajectory traversal may also be modified with both the LFO and MOD LFO, with programmable sensitivities.
Pitch of oscillators may be modified by both LFOs with programmable sensitivities.
These LFO trajectory interactions are great for both slow movement and for crazy MOD-driven wiggling
Oscillator B may be manually detuned / uptuned by +- 12 semitones, with exponential control for fine detune
Oscillators are MIP-map antialiased, the octaves generated by FFT / de-FFT, and there's a secret sauce trick that can be manually enabled to preserve the phase of certain harmonics whilst retaining the critical (in my implementation anyway) feature of a positive-moving zero transition at table location 0 for all octaves.

There is a back-end LPF, which is a trivial 1st order thing but it does go truly transparent at half Nyquist. This is just one filter per polysynth, effectively a very coarse tone control to tune out the psychotic edge of the more extreme waves and bring in some 'mellow', if mellow is what you fancy. This is nice on lush ensemble stringy sounds to push them more background but leave their tones prominent to reinforce a melody.

There are some 21 built-in wavetables - I need to count, there may be just 17 - there are more than 16, I broke the nice power of 2 limit because I had some I couldn't bear to leave out. These include 3 Caitlin vowels, sampled piano, brass, alto sax, viola, and also analytical waves like saw, sin, square, tri.

The Cray XMP / SIMD programming system  is included, to support user construction of rich new wavetables from combinations of existing tables, be they built in or user-programmed.

Nice feature set, eh?

A first generation Raspberry Pi seems to be able to generate about 20 notes of polyphony from this synth, which is a lot of synthesis. So a completely reasonable thing is to create 4 PIANA Virtual Analog monosynths, 3 PIANATRON sampled synths and a pair of 4 note polyphonic PICYCLE wavetable synths, and let them go play.

Obviously on a Raspberry Pi 2 you can do all this and find a cure for the common cold in parallel using neat protein folding tricks. Or just fire up another dozen synths and go really crazy.

This thing sounds even cooler than I imagined, and to be honest I always expected it to sound cool. But it is BRILLIANT. Seriously.

Video to follow.

p.s fave trick to date - the Gordon Reid 'strings' trick with a Slightly Phase Distorted pair of different stacked sawtooths, one FMd by a 3Hz 0.08 semitone amplitude S/H LFO. Lush ...

Thursday, 16 April 2015

Wavetable synth programming - Cray-XMP style!

OK, that's stretching it a wee bit, but bear with me.

I wanted to expose a programming interface for wavetables that would let me ship with a relatively small number of tables installed - less than 20 - and let users construct their own wavetables of arbitrary richness and complexity via manipulation and combination of these 'primitive' tables. This would also mean that I didn't (necessarily) need to support an 'import wavetable' function with all the complexity of that - like having to publish a separate wave building tool etc.

Now, because the wavetables are 1024 samples wide, it turns out the simplest programming interface that I could think of for this was to treat an individual wavetable as a 'register', and perform only vector operations, 1024 at a time. And use a stack-based system so I can be groovy and reverse Polish. i.e. so I don't have to go to the effort of writing a parser.

So the instruction set for wavetable programming consists of this lot -

void init       (); // const reg = 1.0, zero out waves, zero out stack
void loadWave   ( const int slot, const short *samples );
void pop        ( short *here );    // retrieve results of all this work
void pushConst  ( const float c );  // const reg = c
void pushWave   ( const int slot ); // SP++ TOS = waves[slot]
void filter     ();  // take const as normalized freq and LP filter
void shiftLeft  ();  // take const as phase delta and rotate samples left
void resample   ();  // take const as 'speedup' factor and resample
void scale      ();  // stack[sp] = stack[sp] * const
void PD         ();  // phase distort stack[sp] by const
void offset     ();  // stack[sp] = stack[sp] + const
void triscale   ();  // stack[sp] *= triangular window for formants
void sinscale   ();  // stack[sp] *= sin window for formants
void mul        (); // stack[sp-1] *= stack[sp] sp--
void add        (); // stack[sp-1] += stack[sp] sp--

void sub        (); // stack[sp-1] =  stack[sp]-stack[sp-1] sp--

plus a trivial lexer that takes one line of text, strtoks it, and performs switch statements to select which operation to perform.  There are 4 vector 'registers' holding waves, so a new waveform can only be built from 4 others, but of course you can build temporaries then use those as subsequent building blocks.

And it's actually brilliant. I just threw away a couple dozen built-in waves - things like 8 differently phase-distorted squares and sines, and stacked saws with resonant formanty things on top - and in doing so simplified the innards of synth setup, and added this vector programmable machine to let me build these waves outside the application. And now I can craft great big wavetable noises without recompiling. About time too.

Thinking about it, it's a giant SIMD machine rather than a vector processor. Having once spent almost 2 years coding up a 3D pipeline and shader system on an 8,192-way SIMD machine you would have thought I might actually remember that key difference ...

Sunday, 29 March 2015

Wavetable update ...

I may need to grab a bit more video. I just tweaked the MIP map generation step, which enters the frequency domain and for each octave table, strips the high frequencies approaching and beyond Nyquist for this octave, then goes back to the time domain - all the while doing some secret sauce to aid with de-clicking and de-artifacting. I thought it was doing a great job already. But it turns out there was a pretty sneaky trick I needed to do in the secret sauce bit, which I only worked out by watching my triangle wave on screen - one of those proper "Eureka!" moments. And suddenly, everything with interesting / out of phase harmonics - the triangle, the various Caitlin vowels - now sound massively sweeter. Which is interesting, because it implies that I CAN hear phase relationships, and people aren't supposed to be able to do that. Not just that I can hear phase relationships but that they really are key to shaping these sounds. But anyway, once I've prepped a handful of interesting presets I shall turn on the video kit again. I'm fascinated in particular by how, over a small handful of notes, the 'Caitlin vowels' really sound like somebody singing Oooh or Aaah. Or at least, really sound like a speech synthesizer Ooohing and Aaahing away.

UPDATE (i.e. the author may have been having a partial senior moment)

Of course there is another issue I should not discount here - out of phase harmonics, blended over a wavetable trajectory, will cancel completely ...


Here's a thought I had today that I want to make a note of before I forget - instead of auto-computing wavetable octaves, what if I let the user supply them? Yes, I'll still have to FFT / de-FFT to kill frequencies beyond Nyquist, but it would let you capture the tonality of an instrument up and down its range. And suddenly you have a 2D surface of tables, indexed in X by frequency, in Y by time. That's a thought to not let go of ...

Friday, 27 March 2015


Finally - sounds. 

Unfortunately there is something amiss with audio quality here - the original on my Mac sounds great, but YouTube has done something to it and the saw sounds ugly. I may repost it with the volume cranked down to see if that helps ... and done, the linked video is now the newer version, which still doesn't sound as good as it should (i.e. as good as it does on my desk before I upload it) but is at least better than it was before. 

But ignoring the slight fizzy oddness with the saw, those swept formants really sound vowely and voicey. And they look great.

Monday, 23 March 2015

PICYCLE is now playable ...

... and it sounds insane. INSANE. The parametric sonic space is enormous for this thing, it's just ridiculous.

Sunday, 15 March 2015

25% CPU free

Being Boiled :

Endless repeat of 'worst section', all synths including lots of handclap.

Pi Model B+ single-core ARM11 950MHz
6 synths active
4 delays active (of 8 total)
reverb active (post-delay wet signal also has 'reverb send' amount)
sequencer active (separate application)
wave information published to sequencer via shared memory
wave display by sequencer @720p

Free CPU : thrash1 + thrash2 = 25%

I love that the sequencer is 0.3% of the CPU - just running 'top -H' is taking 6x as much! Last thing, shake out PICYCLE. Hopefully will get sounds before tonight's Toad Patrol ...

op - 13:04:39 up  3:03,  2 users,  load average: 2.67, 1.48, 2.25
Threads:  90 total,   3 running,  87 sleeping,   0 stopped,   0 zombie
%Cpu(s): 64.9 us, 33.2 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  1.9 si,  0.0 st
KiB Mem:    445740 total,   289020 used,   156720 free,    16560 buffers
KiB Swap:   102396 total,        0 used,   102396 free,   152432 cached

 3848 root      -7   0  118m  88m 3828 S  55.5 20.3   0:39.26 synth servic
 3853 root      20   0  118m  88m 3828 R  12.4 20.3   0:09.31 thrash2
 3852 root      20   0  118m  88m 3828 R  12.1 20.3   0:09.31 thrash1
 3857 root      -2   0 41004 5860 3144 S   8.5  1.3   0:05.94 glRender
 3846 root      -9   0  118m  88m 3828 S   6.0 20.3   0:04.34 nativeAudio
 3707 pi        20   0  4708 2420 2016 R   1.9  0.5   0:19.17 top
 3847 root      -8   0  118m  88m 3828 S   1.9 20.3   0:01.25 mixer
   31 root       1 -19     0    0    0 S   0.5  0.0   0:48.32 VCHIQr-0
 3858 root      -2   0 41004 5860 3144 S   0.5  1.3   0:00.39 VCHIQ completio
   30 root       1 -19     0    0    0 S   0.3  0.0   0:21.86 VCHIQ-0
 1539 root      20   0  1756 1096 1004 S   0.3  0.2   0:09.67 ifplugd
 3692 pi        20   0  9268 3524 2936 S   0.3  0.8   0:01.86 sshd
 3844 root      20   0  118m  88m 3828 S   0.3 20.3   0:00.19 WiFiprintf
 3854 root      20   0  118m  88m 3828 S   0.3 20.3   0:00.13 rawkeys
 3856 root      -3   0 41004 5860 3144 S   0.3  1.3   0:00.31 sequencer

Saturday, 14 March 2015

Pi day?

Nope. Totally, utterly not.

Pi day would be the 3rd of Quatorzember (or whatever the name of 14th month would be), not the 14th of March. Seriously, I'm British and proud of it, and today is 14.3 not 3.14

So expect an announcement here on the 22nd July. You know - 22/7

Of course it may only be an approximate announcement, or an approximate date, but watch this space. Things are happening.

p.s. this is cool - and demonstrates that an announcement on the 22nd of July is INHERENTLY A GREATER THING than one on the 14th March. Take that, you date-format-challenged, day-month inverted freaks!!