A few months ago, I added an on-board synthesizer to the Mosaichord, my Teensy-based microtonal MIDI controller. I thought I'd post some notes here about how that went, what worked and what didn't, it case it's useful to anyone else doing the same thing, or to anyone working on improving the Teensy audio library.
Here's more information on the Mosaichord: https://desideratasystems.com/mosaichord.html
(Basically, it's a 4-octave keyboard using a 28 note scale. The notes are taken from 7-limit just intonation. The keys themselves are pressure sensitive, and use force-sensitive resistors. I can scan the whole keybed about 500 times per second.)
Here's what the end result sounds like:
Sounce code and user manual can be found here: https://github.com/jimsnow/microtonal-controller
The Mosaichord is something I've been working on for awhile. For the last few years I've been using it as a MIDI controller with a variety of external synthesizers. This works well enough, but I also wanted an internal sound engine so a) I wouldn't have to plug it into some other external device just to get sound and b) I can configure the sound engine exactly in a way that works best with the capabilities of the Mosaichord.
I was able to get a pretty good result without spending a lot of time on it. It sounds like what you'd expect a simple analog-modelling synthesizer to sound like.
Some of the details about how I did it follow:
The DAC
The output device I'm using is the PCM5102a DAC from Texas Instruments, which uses the i2s bus. One nice thing about the 5102a is that it can generate the SCK clock signal itself using a PLL, which frees up a pin on the Teensy. It's just a line output; it's not supposed to be able to drive headphones, but I've found that in practice it actually can.
For a long time, the Mosaichord firmware just used the DAC to output a sine wave of the middle note (1/1) on the keyboard. This was a pretty easy to do using the Teensy audio system design tool. Basically just connect a "sine" object to both channels of an "i2s2" object, and add some code to set the frequency on startup and anytime it transposes to a different key. (The Mosaichord has transpose buttons for +/- an octave and +/- a 100-cent semitone.) This verified that the DAC worked as expected and is occasionally useful for tuning analog oscillators if I'm using the Mosaichord to control a eurorack setup. (The first PCB rev with the DAC didn't work because I neglected to connect one of the pins to ground that was supposed to be grounded, but it's worked fine ever since I fixed that.)
Polyphony and the Audio Design Tool
The Mosaichord is intended to be a polyphonic instrument, so what I want is a certain "voice circuit" and then replicate it a bunch of times, and feed the outputs into a mixer. The audio system design tool doesn't really have any notion of "duplicate this thing N times", so I used the design tool to make one voice, then refactored the resulting code so that I was creating arrays of objects rather than a single one, and putting initialization in a loop.
Waveforms
The basic architecture I'm using is subtractive synthesis. Basically, start with a basic waveform (saw, triangle, square, sine, etc...), modulate the oscillator's volume by key pressure, and run the output of that into a resonant low-pass filter.
I initially used the regular variants of the oscillators (e.g. WAVEFORM_SAWTOOTH), but they sounded very "ring-mody" in high octaves. Switching over to the bandwidth limited versions (e.g. WAVEFORM_BANDLIMIT_SAWTOOTH) fixed that.
Volume slew
I noticed that if I use an oscillator without a lot of harmonics like sine wave, there's an audible white noise sound when I module the volume. (Volume is modulated by key pressure.) I'd run into the same problem recently using a MIDI-to-CV device with the Mosaichord, and in both cases the cause is the same: making abrupt changes to volume causes a stair-stepping effect that's audible as noise.
For the Mosaichord I decided to control the volume at the oscillator rather than the mixer. I don't think it matters a great deal which I chose, but the issue in either case is that I need to apply some smoothing to the volume. And I can't do it without editing the internals of either the oscillator or the mixer because controlling the volume externally can only be done once per "block" (I don't remember how big a block is, but I think it's somewhere in the neighborhood of 32 samples or so) whereas I want it to be smoothed at a per-sample granularity.
I ended up just making a copy of AudioSynthWaveform and adding a slew limit to the volume that I called TaperedAudioSynthWaveform. I would rather have just derived a new class from AudioSynthWaveform, but there was some technical reason why that didn't work. (I don't remember what it was exactly, but probably along the lines of I needed to override the "amplitude" method but it isn't virtual, and I needed access to some of the private variables.)
My slew limit algorithm could probably be better, but after a few tries I got something that made the noise far less noticeable and I figured it was good enough for now.
The audio library could probably benefit from having configurable slew limits on all kinds of settings, but especially oscillator volume, mixer channel volume, filter cutoff frequency and resonance.
Filters
For my resonant low-pass filter I'm using AudioFilterBiquad. So far it's working pretty well. I'm just using stage 0, and use a frequency value between 20.0f and 20000.0f, and a resonance value from 0.1f to 5.0f.
Mixers
I give each voice its own dedicated mixer (so I can easily add sub-oscillators or whatever later). The Teensy audio library has a limit of 4 inputs per mixer and I'm using 16 voices, so the voice outputs get sent to 4 mixers that get mixed down into a single output by another mixer. If I wanted more voices, I would have to add another layer to the tree.
It would be very nice if there were a mixer object with more inputs so we wouldn't have to construct a tree anytime we need more than 4.
Reverb
After that last mixer, the output gets sent into a reverb. I initially used freeverb, but it was kind of underwhelming. I eventually switched over to using the plate reverb from Piotr Zapart, which sounds reasonably good. It has a bunch of parameters that I haven't spent much time tweaking.
I added a final mixer pair (one each for right and left), and feed into those the (mono) dry signal and the wet signals from the right and left reverb channels, respectively. One of the Mosaichord knobs fades from full dry to full wet by adjusting those final mixer input gain settings. The reverb output is not very loud, so I give it an extra 3x gain.
Karplus-Strong
I tried using the AudioSynthKarplusStrong oscillator, but gave up on that for now because the tuning is not very accurate. (Basically, its wavelength is determined by the number of samples at 44.1khz, and that has to be an integer.) It also has no way to pitch-bend a note after it begins, which isn't a total deal-breaker but it is a significant limitation.
I really like Karplus Strong oscillators in general, so I hope we get a better implementation in the Teensy audio library eventually. I made an attempt at improving the code myself, but I didn't really get anywhere with it.
Other non-sound-library details
Having an integrated sound engine gives me more of a reason to care about saving settings that can persist across reboots. I haven't finished implementing it yet, but my plan is to use LittleFS and the built-in flash memory.
Ideally one should be able to export settings to a computer, and the LittleFS library does support MTP, but unfortunately there isn't an option in the Arduino IDE (under Tools->USB Type) for "MIDI + Serial + MTP". I understand I can add that setting myself by editing the Arduino IDE's configuration, but for this project I want end users to be able to recompile the source code with just a stock installation of Arduino and Teensyduino. It would be nice if that were one of the built-in options.
Another random suggestion I have for improving the MIDI library is to make running status a setting you can toggle on and off at runtime. So far I haven't run into any synths that are confused by running status, but if I did it would be a shame to basically have to turn it off for all synths to get that one synth to work.
Here's more information on the Mosaichord: https://desideratasystems.com/mosaichord.html
(Basically, it's a 4-octave keyboard using a 28 note scale. The notes are taken from 7-limit just intonation. The keys themselves are pressure sensitive, and use force-sensitive resistors. I can scan the whole keybed about 500 times per second.)
Here's what the end result sounds like:
Sounce code and user manual can be found here: https://github.com/jimsnow/microtonal-controller
The Mosaichord is something I've been working on for awhile. For the last few years I've been using it as a MIDI controller with a variety of external synthesizers. This works well enough, but I also wanted an internal sound engine so a) I wouldn't have to plug it into some other external device just to get sound and b) I can configure the sound engine exactly in a way that works best with the capabilities of the Mosaichord.
I was able to get a pretty good result without spending a lot of time on it. It sounds like what you'd expect a simple analog-modelling synthesizer to sound like.
Some of the details about how I did it follow:
The DAC
The output device I'm using is the PCM5102a DAC from Texas Instruments, which uses the i2s bus. One nice thing about the 5102a is that it can generate the SCK clock signal itself using a PLL, which frees up a pin on the Teensy. It's just a line output; it's not supposed to be able to drive headphones, but I've found that in practice it actually can.
For a long time, the Mosaichord firmware just used the DAC to output a sine wave of the middle note (1/1) on the keyboard. This was a pretty easy to do using the Teensy audio system design tool. Basically just connect a "sine" object to both channels of an "i2s2" object, and add some code to set the frequency on startup and anytime it transposes to a different key. (The Mosaichord has transpose buttons for +/- an octave and +/- a 100-cent semitone.) This verified that the DAC worked as expected and is occasionally useful for tuning analog oscillators if I'm using the Mosaichord to control a eurorack setup. (The first PCB rev with the DAC didn't work because I neglected to connect one of the pins to ground that was supposed to be grounded, but it's worked fine ever since I fixed that.)
Polyphony and the Audio Design Tool
The Mosaichord is intended to be a polyphonic instrument, so what I want is a certain "voice circuit" and then replicate it a bunch of times, and feed the outputs into a mixer. The audio system design tool doesn't really have any notion of "duplicate this thing N times", so I used the design tool to make one voice, then refactored the resulting code so that I was creating arrays of objects rather than a single one, and putting initialization in a loop.
Waveforms
The basic architecture I'm using is subtractive synthesis. Basically, start with a basic waveform (saw, triangle, square, sine, etc...), modulate the oscillator's volume by key pressure, and run the output of that into a resonant low-pass filter.
I initially used the regular variants of the oscillators (e.g. WAVEFORM_SAWTOOTH), but they sounded very "ring-mody" in high octaves. Switching over to the bandwidth limited versions (e.g. WAVEFORM_BANDLIMIT_SAWTOOTH) fixed that.
Volume slew
I noticed that if I use an oscillator without a lot of harmonics like sine wave, there's an audible white noise sound when I module the volume. (Volume is modulated by key pressure.) I'd run into the same problem recently using a MIDI-to-CV device with the Mosaichord, and in both cases the cause is the same: making abrupt changes to volume causes a stair-stepping effect that's audible as noise.
For the Mosaichord I decided to control the volume at the oscillator rather than the mixer. I don't think it matters a great deal which I chose, but the issue in either case is that I need to apply some smoothing to the volume. And I can't do it without editing the internals of either the oscillator or the mixer because controlling the volume externally can only be done once per "block" (I don't remember how big a block is, but I think it's somewhere in the neighborhood of 32 samples or so) whereas I want it to be smoothed at a per-sample granularity.
I ended up just making a copy of AudioSynthWaveform and adding a slew limit to the volume that I called TaperedAudioSynthWaveform. I would rather have just derived a new class from AudioSynthWaveform, but there was some technical reason why that didn't work. (I don't remember what it was exactly, but probably along the lines of I needed to override the "amplitude" method but it isn't virtual, and I needed access to some of the private variables.)
My slew limit algorithm could probably be better, but after a few tries I got something that made the noise far less noticeable and I figured it was good enough for now.
The audio library could probably benefit from having configurable slew limits on all kinds of settings, but especially oscillator volume, mixer channel volume, filter cutoff frequency and resonance.
Filters
For my resonant low-pass filter I'm using AudioFilterBiquad. So far it's working pretty well. I'm just using stage 0, and use a frequency value between 20.0f and 20000.0f, and a resonance value from 0.1f to 5.0f.
Mixers
I give each voice its own dedicated mixer (so I can easily add sub-oscillators or whatever later). The Teensy audio library has a limit of 4 inputs per mixer and I'm using 16 voices, so the voice outputs get sent to 4 mixers that get mixed down into a single output by another mixer. If I wanted more voices, I would have to add another layer to the tree.
It would be very nice if there were a mixer object with more inputs so we wouldn't have to construct a tree anytime we need more than 4.
Reverb
After that last mixer, the output gets sent into a reverb. I initially used freeverb, but it was kind of underwhelming. I eventually switched over to using the plate reverb from Piotr Zapart, which sounds reasonably good. It has a bunch of parameters that I haven't spent much time tweaking.
I added a final mixer pair (one each for right and left), and feed into those the (mono) dry signal and the wet signals from the right and left reverb channels, respectively. One of the Mosaichord knobs fades from full dry to full wet by adjusting those final mixer input gain settings. The reverb output is not very loud, so I give it an extra 3x gain.
Karplus-Strong
I tried using the AudioSynthKarplusStrong oscillator, but gave up on that for now because the tuning is not very accurate. (Basically, its wavelength is determined by the number of samples at 44.1khz, and that has to be an integer.) It also has no way to pitch-bend a note after it begins, which isn't a total deal-breaker but it is a significant limitation.
I really like Karplus Strong oscillators in general, so I hope we get a better implementation in the Teensy audio library eventually. I made an attempt at improving the code myself, but I didn't really get anywhere with it.
Other non-sound-library details
Having an integrated sound engine gives me more of a reason to care about saving settings that can persist across reboots. I haven't finished implementing it yet, but my plan is to use LittleFS and the built-in flash memory.
Ideally one should be able to export settings to a computer, and the LittleFS library does support MTP, but unfortunately there isn't an option in the Arduino IDE (under Tools->USB Type) for "MIDI + Serial + MTP". I understand I can add that setting myself by editing the Arduino IDE's configuration, but for this project I want end users to be able to recompile the source code with just a stock installation of Arduino and Teensyduino. It would be nice if that were one of the built-in options.
Another random suggestion I have for improving the MIDI library is to make running status a setting you can toggle on and off at runtime. So far I haven't run into any synths that are confused by running status, but if I did it would be a shame to basically have to turn it off for all synths to get that one synth to work.
C++:
struct MidiLocalSettings : public MIDI_NAMESPACE::DefaultSettings {
static const bool UseRunningStatus = true; /* avoid re-sending status byte when it hasn't changed */
static const unsigned SysExMaxSize = 10; /* we don't expect to receive sysex */
};
MIDI_CREATE_CUSTOM_INSTANCE(HardwareSerial, Serial5, dinMidi, MidiLocalSettings);
, and inspect my changes to see if they’re equivalent to the ones you posted. They won’t be identical due to the fixes for loss of precision, but they look very similar.