Teensy-LC, artificial neural network, usb midi synthesizer.

Status
Not open for further replies.

ohnoitsaninja

Well-known member

I've been working on this for a while, but it's sat dormant for too long so I needed to get this out in the world before it's forgotten in the back of a drawer. I'm not publishing all the code now, as there are several programs involved and it all needs to be cleaned up better. I have an alternative NN trainer program in C with AVX2 and multithreading too that trains much faster, but the processing sketch has some nice graphics. What I can give out if anyone wants is the base code for just a teensy-LC usb-midi synth, as I have not found any complete examples anywhere yet. The ANN code is a heavily modified version of what's here. The ANN is optimized for fixed point 32-bit ints. More info in youtube description and annotations.
It was a very fun project, learned a lot, and I hope to expand on it greatly some day.

To be completely honest, the only reason I used teensy instead of making into a VST is because I couldn't get the VST sdk worked out and compiling. I had some LCs laying around, and I knew that it could work. It should say something about the ease of use with programming teensy and the excellent libraries, when it's easier to program an embedded synth than one inside your desktop.

While there are both artificial and real neural networks that have been used to compose music on other instruments, as far as I have been able to find, this is the first endeavor towards an actual neural network based instrument.

While I'm here, there was a few issues I ran into that I haven't been able to figure out. The DAC is 12-bit, so 0-4096, it would make sense for my AC signal to bounce around 2048, I have never been able to get it to work properly this way, however treating it like 11-bit and using 1024 as my virtual ac ground works fine. A bigger problem, depending on how ableton is setup to send usbmidi data, it can completely overwhelm the teensylc with messages to the point where it stops working, there isn't a usbMIDI.flush() and Serial.flush() doesn't help, adding more while(usbMIDI.read()) throughout the code didn't help either.

Source for a basic Teensy-LC Synthesizer
https://github.com/ohnoitsaninja/Teensy-LC-basic-usbMidi-synth/blob/master/basicSynth.ino
 
Last edited:
I'm at work so I can't view your video ... sounds really interesting .... A teensy LC usb-midi synth sounds interesting! Details?? I guess it uses the onboard DAC ... and is controlled via usb midi .... could it be cut over to serial midi, so I could use it with my oldschool master keyboard? What kind of sounds are you getting ..... Oh, I feel like quitting work right now ... I was thinking about putting some space sounds into a song of mine last night (I have a roland rack synth / preset box) but some rude low-bit sounds from a teensy might be the ticket.
 
Last edited:
Because it's the most minimal of neural network synth engines right now, it sounds very well behaved(too well?). This is mostly attributed to the waveIndex being driven by a saw wave at the desired note frequency and lack of an amp envelope. So when it's being tuned or different notes are selected, the neural network doesn't really even see any of that. I think if I could somehow get the clock, tuning, and envelope all inside the ANN instead of C++, it will start sounding much more organic. The target waveforms can be changed to anything, but the sequence it currently is set for sounds very much like a low pass filter on a square wave.

My experiments with just a plain/normal LC synth had surprisingly good quality, quite a bit better than what I'd refer to as lo-fi IMHO. I never measured the actual bit depth tbh, based off experience with bit crushers it sounded higher than 9 bits.

As far as adding serial midi, I have an old sparkfun arduino midi shield that would probably be trivial to get working, pretty sure the only components on the board besides midi jacks are optocouplers..

Here is my plain synth source, https://github.com/ohnoitsaninja/Teensy-LC-basic-usbMidi-synth/blob/master/basicSynth.ino It's very minimal, about as compact as I could get it. It should be a good starting point for anyone wanting to make their own LC synth.
 
Last edited:
Sorry for the double post, but I put a bunch more time into programming this. Got the ANN code working even faster, fixed up some audio glitches, added 4 voice polyphony, better interpolation, preset bank for multiple trained NNs, and a sweet 3D visualizer on the training program.

Video shows lots of 3D neural network response, oscilloscope view of teensy-lc DAC, live demo + cat, and a more polished short song at the end.


Here is a look into the inner workings of a trained synth neural net.
https://www.youtube.com/watch?v=tJcweLqgIUE
 
Last edited:
Status
Not open for further replies.
Back
Top