PDA

View Full Version : Playtune: MIDI to sample-based audio synthesis



LenShustek
08-16-2016, 01:54 AM
About five years ago I reimplemented, for Arduino, some code I had written 40 years ago for the then-new Intel 8008 microprocessor to play polyphonic music with square-wave generators.
https://github.com/LenShustek/arduino-playtune
For the original version I had to type in the score, but it was easier decades later to write a "compiler" that reads MIDI files.
https://github.com/lenshustek/miditones.

I'm now a Teensy fan, so I thought it would be fun to use its D-to-A converter to make a version of the player that generates better sound by using sampled waves of real instruments. This is probably duplicative of other many other synthesizer projects, but I had fun and some of you might enjoy browsing the code.
https://github.com/LenShustek/Playtune_samp

-- Len Shustek

oddson
08-17-2016, 01:44 AM
I thought the name looked familiar
https://en.m.wikipedia.org/wiki/Len_Shustek

LenShustek
08-18-2016, 02:46 PM
Guilty as charged. In particular, of being an old-timer. I've been writing computer programs for over 50 years, and I enjoy it as much now as the day I started.

oddson
08-18-2016, 10:47 PM
After leaving the faculty, he co-founded Nestar Systems in 1979, and Network General in 1986. In 2003, he provided a $2.5 million endowment for the "Leonard J. Shustek Distinguished Professor of Computer Science Chair" at Polytechnic Institute of New York University.
$2.5million eh?....so did you see Paul's kickstarter... https://www.kickstarter.com/projects/paulstoffregen/teensy-35-and-36 ;)

Frank B
08-18-2016, 10:59 PM
Looks interesting!
Have you considered to use the audio library?

LenShustek
08-19-2016, 12:20 AM
I didn't know about the Teensy 3.5 & 3.6 project. I just backed it. And yes, I'll probably start playing with the audio library; it looks great.

LenShustek
08-20-2016, 05:15 PM
The Audio Library is impressive! I'm surprised it's as fast as it is, given that it uses some floating point. In my MIDI music players for the 8-bit AVR-based Auduinos, I had to avoid floating point and even multiply/divide in the realtime parts.

I'm going to find time to elaborate the PlaySynthMusic example (which was evidently based on my Miditones-generated Playtune bytestream) to use recorded instrument and percussion samples like Playtune_samp.

It would be nice to rework it into a synth-type music object (like AudioSynthWaveform) that you feed a Playtune bytestream to, so that it could easily be included in a bigger project. One difficulty is that I don't see an easy way for new audio objects to internally use other existing audio objects like AudioEffectEnvelope. Am I missing something? I suppose I could duplicate the functionality inside the new object. Or I could instantiate private copies of the existing objects and shuttle audio data blocks between them like AudioStream does for an AudioConnection. Is that the right approach? Are there examples of that technique that I should look at?

LenShustek
08-24-2016, 04:00 AM
I didn't get any guidance from folks here, so I chose the first option: duplicate all the functionality inside the new audio object.

I've created a new audio synthesis object called synth_Playtune that reads an encoded "Playtune" bytestream created from a MIDI file. It plays up to 16 simultaneous sound generators that are internally mixed to produce one monophonic output stream.

Sounds are created from sampled one-cycle waveforms for any number of instruments, each with its own attack-hold-decay-sustain-release envelope. Percussion sounds (from MIDI channel 10) are generated from longer sampled waveforms of a complete instrument strike. Each generator's volume is independently adjusted according to the MIDI velocity of the note being played, then all the channels are mixed.

If you want to hear a sample output, see https://www.dropbox.com/s/2pgr3g0ls6z0osj/MoneyMoney_env.mp3?dl=0. The source code (which is "beta test" level) is at https://github.com/LenShustek/Playtune_synth.

Reggie
08-24-2016, 11:52 AM
Hi Len, I had a listen to your mp3, it sounds really cool, the timing is perfect, I wish I knew more about the audio library to help out but by the looks of things you've got a good handle on it already!

PaulStoffregen
08-24-2016, 11:34 PM
The Audio Library is impressive! I'm surprised it's as fast as it is, given that it uses some floating point.

Internally, it's all fast integers. Floats are only for the Arduino API. Even then, most of those use inline functions for the float to integer conversion, so for the common case of compile-time constants, the compiler optimizes away all floating point code.



It would be nice to rework it into a synth-type music object (like AudioSynthWaveform) that you feed a Playtune bytestream to, so that it could easily be included in a bigger project.

That does sound interesting. Please remind me of this after we've shipped all the Kickstarter rewards. Until then, I'm on a strict no-new-projects diet!

LenShustek
08-25-2016, 05:49 AM
Well, synth_Playtune is ready for you to check out whenever you come up for air. We're all looking forward to a 180 Mhz Teensy!