yet another Teensy based MIDI Sequencer

[Uija]

Hey!

I like to present the project, I am working on currently to your.
it is a MIDI Step Sequencer running with a Teensy 4.1 with 16MB PSRAM.

I cannot count the times I started working on my own Sequencer (first one on Atmel chips before Arduino was a thing), but did my first real project around 2 years ago, after I realized how cheap you can order PCBs and you don’t have to use those breadboard where you need to cable every connection or build the PCB yourself.

Tried several approaches to Sequencing and after a very modulate x0x one, that I could arrange in many ways, I committed on a layout. After ordering the PCBs I realized that is is way more inspired by a Cirklon than I thought. It is basically a Cirklon with one additional row of buttons in a smaller form factor.



Hardware First:
Teensy 4.1
53x MX style mechanical keys
19x Rotary Encoders with momentary switches
69x WS2812B RGB LEDs
9x Multiplexers for the Buttons
2x I2C Port Extensions (MCP23017) for the Encoders.
A 256x64px OLED Display that I found as replacement part for some HI-FI -gear in a french onlineshop.
a 3D Printed Case (380x114mm), 3d Printed Keycaps and a CNC routed front panel
4 DIN MIDI in and out. 4 „devices“ as USB Device and up to 4 connected devices to the Host port.



The sequencer is a 16 Track sequencer (could be more, but currently I don’t need more and those 16 buttons make 16 so easily accessible). 16 Patterns per Track (again, the number of buttons), „unlimited“ number of bars per pattern, with between 1 and 16 steps per bar. Every pattern can be it’s own type (Mono, Poly, Drum and a few Generative experiments) and resolution.
On mono patterns, there is one note per step with notevalue, velocity, length, offset (to move it on the 192ppqn grid), note repeats, Propability, up to 4 CC values and the possibility to chose a chord and a inversion.
Drum and Poly patterns are just a list of events on a timeline. as the UI is based on Steps, notes are entered on those steps, it can moved around and be recorded freely. Every note has its individual configuration (like a mono step without the CC).
CC can be places the same way as notes (and recorded that way), or drawn like Modulation curves in a DAW. i like this approach more, for most of the stuff, because as slow as midi is, too many events can „overflow“ the speed of MIDI.
On top of that, every Track has 4 Modulation Lanes, that are lfos (and a separate Random area, for making velocity and timing less robotic with just a few clicks.

If wanted, every can be forced to a scale, that is defined on the project.

Currently the bars are played one after another in the order they appear in the UI. As every bar has its own length and can be repeated a number of times, possibilities are big here. I tried to make this more flexible, but that became a UI ‚and usability mess for something that sounded cool, but I don’t really need.

Tracks hold „instruments“ that are basically configurations what output is used. MIDI Port(s), Channel(s), a „base note“ that mono patterns have as default note), if it can receive Bank Select and Program Changes, if it should be forced to scale if the project enables that. instruments have a number of multi-timbral parts. if that’s there, on the Track page a Channel-Field appears where the wanted channel is selected.
For Drum-Instruments, the midi port and channel, as well as a Note and a name are stored to the ROW, so every individual percussion instrument can be freely configured. Some drum machines have one instrument per note, others have a midi Channel per instrument, and with adding the port to it, I am able to send single Percussions to different Sound sources.

Last there is a Songmode that is quite similar (and easy) as the Elektron ones.
An „unlimited“ number of rows are played In order. Every row chooses a pattern (or none) per track, a length, a tempo, a transpose value and a scale, that everything can be forced into.

Currently MIDI recording is the least loved part. the reason why I spent so many time in trying out every sequencer out there etc. is because I am way better in programming notes than playing them.

currently I am rewriting most of the firmware, to get rid of parts of the old hardware as well as design decisions that are based on sequencing- and hardware-concepts that I threw away. I would say I am at 90% of where I was before. Still missing is the MIDI FX section, that contained a MIDI delay (playing back notes via midi but with reduces velocity to emulate delay) and an Arpeggiator.

Everything is can be stored on the SD Card (and loaded from there). it’s writing it’s own binary format, so save time and space. loading is fast enough so with a little work I could make them load and play without the listener noticing, but as I am not performing, I don’t need that.
With 16Mb psram and most of the memory dynamically allocated, there should be enough to load two projects to switch between them.
As the MIDI Sequencing itself is running in a IntervalTimer, loading, saving, IO and Display-Rendering is not effecting the MIDI performance at all.
The MIDI clock is very tight. I wish, all of the hardware would be that tight…

I will try to make and post better photos and maybe later a video, when I have better light in my room some day.

 

[MrCanvas]

Looks impressive and very well thought out! I can't wait to see more pictures and videos of it in action.

Also secretly wishing for a code share because it sounds like you put a lot of thought in to timing and optimization. In my project I just put everything critical on an intervalTimer and hoped for the best.

 

[Uija]

Thanks!

That’s sounds more rocket science than it is: to summarize my upcoming wall of text: I just send the events first and do the evaluation second.
On one point I redid how I evaluate the midi events. what I did was, that every pulse I check if the datastructure that holds everything has notes to play for me, if there are events from the past I need to look at (for note repeats, MIDI Delay etc), Generated the MIDI for that and send it out.
I didn’t like that, mostly because that „keeping track“ part got more complicated with every new idea I had.
I changed that in a way, that every event has a delay-time. Is it 0, the event is sent out, if it’s bigger, it’s stored and sent later. I already was tracking time, to know when to send note offs, so it was not something special.
more by exident than by searching I found a bug, that caused everything to fire one pulse late. I did not notice that, and I wouldn’t have looked for it. Or would I have known of it existence, if I didn’t see it when looking at the code. I did tests with switching that around, and I didn’t feel a different. how could I, everything that happened was, that me hitting play took 2.5ms (at 120bpm) to start the playing. I measured how much I suck in hitting play on the beat and found out, that I am more off most of the time, so I just kept it that way (but now implemented in a way, that I could just switch that off instead of relying on a bug. By beeing able to just start everything late, I am also able to do a little bit of latency correction if I had hardware that might be late. And if I would need more than 5ms, I could just „drop“ 2 pulses on play and do the evaluation 2 pulses ahead of time.
I am quite sure, no one would notice that, luckily it’s just me using it

With this accident I switched from doing as much as I can to be as fast with preparing everything every pulse to: I love to have code as fast nice as I could, but theoretically I just need to make sure, to be done with everything before the next pulse needs to be sent.

P.S.: As I am able to move around notes on the grid in that 192ppqn-resolution I can easily play around when I notice notes beeing early or late. even with short transients, that I move around, it takes more than 1 pulse to be noticeable. on everything non theoretical (sounds I use) I can start to hear Phase issues later.

 

[neutron7]

Your "Circlone" is amazing!

Here is a Cirklon for people who haven't seen, it is a really good sequencer with a long waiting list.

.

 

[Uija]

Yeah, I am on the waiting list for a few years now. I joked around Mid 2020, that it might be more accessible and faster to just build your own sequencer, even if you have now clue about electronics (as I did). Didn't know, that I was not that far off.

 

[fdaniels]

Impressive! You really should implement the Cirklons concept of Aux Events, which makes the Cirklon the Cirklon and sets it apart from other sequencers. Sadly its hard to get into it without having a Circlon at hand, the manual gives just a brief glimpse of what you can do with it and sadly doesnt cut it. Yes, i own one (#290).

 

[Uija]

In preparation of "someday I will get a Cirklon" I watched everything about Cirklon on YouTube. There is one that explains the AuxRow Concept quite good but I need to dive deeper into it, as soon as I have all my basic features done.
The way, how you can add CC, Chords and ChordInversions to a Step in the Mono-Pattern is built in a way, that I can add other Parameters to manipulate steps and other parameters.

 

[Uija]

Very unhappy with my video attempts, but here are some photos of the UI:


Projects have Tempo and Scale information, that is used to stay in scale when randomizing note values and for chord generation.
Project also has a global quantization setting that is quest for queuing Pattern changes and Mutes.
It also shows the playhead of every Track.


Track have a type (Drum, Mono, Poly and in work are some generative ones) as well as an Instrument definition. For Instruments that are multitimbral, the midi channel can be adjusted.


Pattern consist of currently up to 16 bars (could be more, it's just a setting) with each up to 16 steps. Every bar can have its own start- and end-step and a number of times it is played until the sequencer is progressing to the next bar. Pattern have a Resolution (from 1/1th to 1/32th defined). Every pattern can have a program and bank selection to change sounds per pattern.
Every step has its independent setting of Notevalue, Velocity, Length, Offset (24 subdivisions per step), Propability (Like on Elektrons), Ratchets and 4 Lanes Modifiers, that are something that you could compare loosely to Aux Events on a Cirklon. Currently it is able to generate chords with inversions, send CC Messages, define the Probability of the next Modifier-Lane and Randomize Notes, Velocity and Octaves. (Last feature is awesome to bring life into 303 Patterns, by staying on the rhythm and melody, but throwing in randomly accents and octave changes).
Pattern can be recorded (Quantized and unquantified) from MIDI.

Drum Tracks have up to 16 rows (Poly tracks have up to 16 note polyphony). Every row can have its own start- and end-step per bar. In the definition file for drum machines, you can name every row and give it its independent MIDI Port, MIDI Channel and Default-Note-Value. That way you could combine 16 single-voice-drumsound-generators into one Track (16 Ports/Channels) or work with drum machines, that have one channel per track (Elektrons for example) or those, that have one Channel, but use a separate Note-Value per Drum Sound (Like the Drum-Engine on the MC-101). You can change the note-value per step, so on drum machines, that are organized by channel, you can play chromatic notes.
Every step on drum tracks also have the same options, than melodic ones, without the Modifiers.

Every Pattern has 8 slots for Modulation. Those can either be LFOs, Graphs or a special Random-Generator that I use for Drums to change pitch and velocity of hi-hats without the overhead of an LFO, as it is always once per trigger.

LFOs are what you expect them to be, with typical Waveforms like Triangle, Sinus, Saw, inversed Saw, Square and Random.

Graphs are one of the main reason I started working on my own hardware devices years ago. LFOs on Synths are awesome, Parameter-Locks on Elektrons too, but I really missed the precision of Modulation-Possibilities in a DAW.
Graphs do that. They have a resolution of 192 values per bar (12 per step on a 1/16th division), and can be up to 32 bars long (again, can be changed, if I need it). You can scroll through the graph with the left encoder and change the value by the right one. While you could define 12 values per Step, the values that are sent to the Synthesizer are calculated on that 192ppqn resolution of the sequencer, thats 4 times per value. Those pulses are interpolated.
As drawing the curves might be tedious, you can record them either from external CC messages, or by reading one of the 2 Potentiometers, that are on top right corner of the device.

 

[Uija]

And because it is then tedious to make changes when there are 192 values per bar, you can simplify the curve afterwards while being able to choose how tolerant it is.

You can go from checking if the value that is set to that position is the same as if you would interpolate it up to giving it a range. I found out, that dependent on the frequency and amplitude of changes a value between 1 and 3 is usable.

I already talked about instrument definitions. Instruments have a Name, a Midi Port, a Midi Channel, Default Note, if they should be ignored, if "Force to Scale" is active, if its a Elektron-Like Device, than Program-Changes are sent one step before the pattern ends. If they are able to receive Program Changes and Bank selects at all.
On top of that, you can define Names for CC Numbers:

As well as Names for Drum rows:



I am also working on generative features. You can already generate Rhythms and Notes:

For melody-tracks, you can either generate new triggers via Euclidean, or reuse existing triggers. Notes are randomized from notes of the given scale, in a defined range of octaves based on weights you can define for every of the 7 scale notes. You can also let the generator randomize velocity on a given range.
For Drum-Tracks, its only a euclidean generator with that randomize-velocity feature.

Last but not least, I wrote a "configuration framework" where I can easily add pages and options to the configuration area. the settings are stored in a json file. While performance-irrelevant section (like UI) just read the data from the JSON Document (ArduinoJSON), I have some values directly accessibly for stuff thats used in the pulse-Interrupt (Settings, what transport informations is sent to what MIDI Port as well as what MIDI Port is enabled for reading).


I am currently working on a "Berlin Style" Arp, Bass and Chord-Progression Generator that is initially generating patterns and than slowly evolving them.

I also love the Matricial Sequencer on the Oxy one and already did a proof of concept in my "framework".

On my list is mostly:
Swing setting per Pattern/Track/Drumrow
MIDI Effects (Arpeggiator, Note follower, MIDI Delay etc.)

As I am generating notes one MIDI Pulse in advanced, I have a lot of room for realtime calculations. Modulation (LFO and Graphs) that are routed to external devices are evaluated and sent, if changed to last value on every pulse currently. Notes are generated once per step, as I queue MIDI Events with a "delay_in_pulses" into my MIDI-Player, that is also handling sending note-offs, managing voices, extending notes in legato etc.). That way I also handle Ratchets.


Besides all what is seen on screen, I do a lot of UI with the LEDs on the Buttons and Encoders (Highlight active Steps, Sequencer Playhead, Buttons that are relevant in the current situation) as well as a ton of overlays that give help informations, context menus, short-success popups etc.






I think all that UI work takes up most of the Code-Memory that is used

Project with everything are saved in a binary format, with patterns being in separate files to reuse them in other projects if wanted.
Loading and saving is working during playback, but while not messing up playback itself, it might sound awful. Have "Queue loading of Projects when Sequencer is running to make switches between songs possible" on my todo list.

I am also working on a way to film me using it while having both: good light and no bad shadows.....
As soon as its done, I can showcase all those features, that are more visible when using it (changing sections of a bar, copy/paste, managing tracks, Mutes, Patterns, Switching, adding, duplicating bars etc.)

 

[Uija]

Added some Live-Performance features like Instant or quantized Mute and Pattern changes. While I tested that I realized, that while it is working somehow, it is not the best solution, if you want stuff to be done fast and maybe do more than just pressing buttons on the sequencer.
A long while back I started tinkering around with the Launchpad Pro mk3, that you can completely control in Developer-Mode and I remembered that I had quite some fun using the LPP as Clip-Launcher in Ableton and Bitwig.

So I sat down today and added some LPP controlling code to the Sequencer. There is an option in the configuration, where you can enable or disable controller mode for the LPP, so you can still use it as MIDI Keyboard (its my main source for playing notes live, those few times I do so). When in controller mode, I can start and stop patterns (quantized and unquantified) as well as Mute and Unmute Tracks (and Drum-Rows for Drumtracks) again, quantized and unquantized.


As I currently have 16 tracks and 16 patterns, you can scroll horizontally and vertically with the arrows. Using the Launch-Arrows on the right I can launch all patterns in that row. In Mute-Mode, the buttons at the bottom (blue) select the active Track.
In Mute-Mode those bottom-tracks mute and unmute the tracks. If the current selected track is a Drum-Track, the bottom 2 rows represent the Drum-row, so I can mute/unmute them separately. And while I was working on it, drum-rows that contain active triggers are highlighted.

Now Jamming is a lot more fun.

 

[Uija]

I redid the song mode the last days. It was a "simple" pattern chain functionality like on the modern Elektron boxes, now it is more a linear timeline like in a daw. A little bit like it is done in the Machine+. You can place patterns on the timeline, give it a length, make it start on other than the first steps etc. That way the tracks are working completely independent of each other in the song mode, and I can let a pattern play on one track without resetting or interrupting it, while changing, and muting etc. other tracks.

 

[MatrixRat]

Totally awesome. I like the usage of the Launchpad, has inspired ditching the current 8x8 NeoTrellis setup, and using one in current project.

Curious about it's current draw when all buttons are lit.

 

[Uija]

My very first "real" sequencer attempt was based around the Neotrellis grid (was finished just when Hapax was announced).


I ended not liking the feel of the neotrellis. The buttons have tons of wobble (because they are quite long) and the travel is so long, that you notice it a lot when playing it as keyboard (After working on v2 and v3 I am not sure anymore if it was really the travel or the way I implemented the IO handling tho). So after checking out how to DIY my own Silikon-Button-Grid and thinking about trying a different approach, I skipped the second version and built the first one of my current sequencer.

Here are some pictures of the very first prototype:

It's quite bigger. I built the controls out of many small PCBs (you have to order 5 per PCB in china, so I designed it in a way, that I use more than one), but I really didn't like all the cable connections. Was a total mess, so the current version has everything (but the MIDI Outs) on one board. I also removed the distance between the buttons, because I wanted it smaller and I didn't need it that way.

I also tried out using a Raspberry Pi 4 as brain, with a way bigger display.



I stripped down the Teensy firmware for it, so it just sends all the IO informations via MIDI to the Raspberry over USB. I could reuse 80% of the code, just had to change the actual drawing of the screen and the midi using Alsa. While I loved the bigger screen, with colors and beeing able to show a playhead etc due to multithreading, I could feel the impact of the OS prioritizing CPU resources. It produced a lot of jitter and latency.

at the end I tried out the typical ILI9341 display first, moved to 128x64 OLED (elektron style) and with a short visit of 800x400 I landed on a 256x64 OLED. I would love to have a little bigger screen, but its really hard to find valid options that are usable in a DIY environment, and I have not IO left to change from SPI to 8bit

 

[MatrixRat]

Just been playing around with a shiny new Launchpad mini and totally get what you say about the NeoTrellis button action.

Need more buttons and what is looking like the flexibility and relative ease of creating the multiple layouts needed to support recent new stuff in the main project without revisiting the NeoTrellis rabbithole. Thank you for the inspiration.

Dwarfs the ILI9431, currently function is more relevant than form but thinking of Raspi screen or maybe
https://www.buydisplay.com/spi-1280x400-7-84-inch-ips-tft-lcd-module-optl-capacitive-touch
for down the track.

 

[BriComp]

I don't know how fast you need to update the display, but have a look at these.
Updated by Serial so only take up 2 I/O lines.
Screen sizes up to 10.1" (for a price!!).

I have a library which eases the use of these displays if you are interested.

 

[Uija]

Hey!

Do you mind sharing some infos? Are you limited to using the elements in their editor? Am I able to build custom elements?
What is the restriction on the refresh rate?

Thanks!

 

[BriComp]

Below is a video of something I did for a Home Heating Boiler Control as part of a larger Home Heating Management System.

The Nextion has a variety of different Objects/Tools that can be displayed/used to control the display/teensy.
If none of those fit then you can load pictures into the Nextion which can have areas of that picture ring fenced to cause an event when pressed.
The pictures can be sent to the back/brought to the front so pressed/not pressed versions can be displayed.

If you download the Nextion Editor you can draw a screen and run it in Simulation Mode to give yourself an idea of what is possible.

Basically you have to MCUs, Teensy and the Nextion, where the Nextion MCU controls/interfaces with the display.

EDIT: The Nextion Serial interface can run at up to 921600 baud though 115200 is more usual.

 

[Uija]

I think I will just try it out. My UI consists of a lot of special graphics. And not many pure Text-Boxes. I will try that out. If not for this project, maybe for another one.

 

[BriComp]

If you have run my video, the switches along the top are "Nextion Dual State Buttons" with one pic for one state and a second pic for the other state.

The pic to be displayed is automatically handled by the Nextion dependant upon whether the Button is "Pressed" or not, i.e. it's a bistable switch.

The x,y,w,h are all under the control of the developer.
It is also possible to "drag" these Buttons.
This feature is available for most of the Nextion items from the "ToolBox".

There is an event for Touch and for Release of the switch.
These events can run "Nextion Code" or send Serial info/commands to the Teensy.

It's NOT all about Text boxes.

BE AWARE that there are various GRADES of Nextion displays. Only the highest grade has all the features/functions.
It also has the most memory and the fastest processor.

You can see/download my Nextion Library here.
If you look in Examples you will find the INO that communicates with the Nextion for the example shown in the Video as well as the Nextion HMI file.

 

[Uija]

yeah, I got that with those pictures, but if you look at that image, you see, what I mean with: I have more custom elements:



The Piano on the left scrolls dependent of the row you currently have selected. With that changing, the background color of the rows change.
The vertical bar is moving (Playhead) and is half transparent. The White boxes there are the notes. On this screen they are simple. On Poly-Screen there can be 16 per column, with different length, semi transparent to see what is under it. On the poly screen there is also a horizontal bar that is half transparent and shows the currently selected note. That is nothing you can do with "simple" image swapping, or that crop feature.

I am currently playing around with the editor, checking what is possible. I hope that the screen isn't refreshing that "ugly", as the simulation is. If that flickering will show up every time I change something, that would be awful.
It is really really sad, that not all elements have a "OnValueChange" Event, so I could just set the value and the display is calculating and rendering some stuff It seems, that I will have to send a lot of draw-commands on some pages.

 

[Uija]

Okay, he we go. The display isn't really made for stuff like my drum and poly Grid. The grid contains out of a few layers. The first is a grey background. The second is per row and represents the area, that is active (somewhere between 16 and 1 grid cells). It is either light or darker color, based by if the row is currently selected.
On top of that are the white lines making out of that blob of colors a grid. And the last one is 16x16 possible boxes, that show if the step is active, or not.

The grey background is just part of the page-background-image.
Because I am not able to modify the width of elements during runtime, I use a progress bar for the rows-layer. It uses images, as I need the background to be transparent. The progress bars are bigger than the screen, so I am able to set numbers exactly matching the 16 boxes. Because I can also define the first step, I move the progress bars on the X axis.
On top of those I placed an image element that contains the grid.
Now I can manipulate the foreground-image of the progress bars to highlight the currently selected row. the progress bars are global, so they don't miss their value so rendering does not jump.

Now I need to be able to show white squares in every cell of the grid. I tried a few approaches. My first idea was to add 16x16 images. As I would be able to access the elements by id, when I would arrange them after another, I would be able to enable and disable them. Well, it seems, that you cannot add 256 elements to a page.
next Idea is to draw them on top via fill command. That works, but it flickers, as the screen is first rendered and then I am drawing on top. I even started to use ref_stop and ref_star to pause the rendering until I am down with drawing, but it seems, that in that case the elements are overwriting the manual draw commands.

So I draw the complete grid by hand via the serial interface. when I use ref_stop and ref_start again, I can do that without noticeable flicker on the screen.

I also tried using the timer, but that shows an empty screen flicker, when I display the screen the first time, because the first call is after the timeout of minimum 50ms. I think I could live with 20fps, but I have around 40fps at 256000bit/s, when I am doing it by hand. It would be cool, if there were a Code-Element, that I can add to the screen and then execute code, that is executed during the page-rendering-process. that would solve all the issues...

I will keep working on it and see, if I am happy with the results am I am able to archive. There are a few issues I have with how text is aligned in Text-fields. It's very asymmetric until the padding is big enough. So again, I might have to draw most of the pages myself, because I have text labels that can sometimes act as edit-fields, that I want to highlight with borders.

But at the end, I really love the concept of a smart display. If I am not happy with how that works out, I either find a way to flash my own firmware to the display or I find a 8bit or 16bit protocol Display, add a simple STM32 chip to it and build my own smart screen, that only needs values sent to it

 

[Uija]

Oh, I forgot to add the image:

 

[clearskyer]

Hi ! Very interesting project ! Trying to do something similar here ! Do you think you could share some of your code ( basics clock, sequence, midi etc.. ) ? Looks like the tightness is great and i would like to start with good fondation. ( im literally new into coding ).. no worry i understand also Thanks !

 

[rcon]

Your project looks awesome! good work!