USBhost_t36 MIDI losing midi note on / off events (Teensy 3.6) during modwheel change

rvh

Well-known member
I'm using the USBhost_t36 lib to connect the popular Arturia minilab mk II midi keyboard/controller to a synth I have developed for Teensy 3.6. Unfortunately note on and off events are lost when I play keys and move the "modwheel" (really a touch slider) quickly at the same time. Interestingly, I see the issue only with the modwheel and not other controllers, including the pitch wheel which is also a touch slider. Maybe the minilab is sending modwheel data at a higher rate, but it seems a bit odd to me that the USBhost_t36 would not be able to keep up with any midi controller. I have tested the synth with the (non-host) USB midi connection, connectng the minilab to a computer and then the computer passing the data on to the Teensy via midiOX, and it works just fine via that midi path.

So the problem appears to be with the USBhost_t36 lib code rather than my software. And indeed I can replicate the same problem using the "InputFunctions" example code provided in the library. I stripped that example down to the bare minimum and left in only the callbacks for midi note on/off and modwheel CC. I added one variable that keeps track of the number of notes being held down (NumNotes) that is incremented for note on events and decremented for note off (or 0-velocity note on) events. When I release all the keys on the controller, that number should then be 0, but it is not always when I move the modwheel quickly whilst playing notes. The serial print shows that both note off and on events can be lost during modwheel movements. It even happens when I do nothing inside the modwheel callback (commenting out its serial print messages).

Any suggestions for things to try are appreciated. I reinstalled the latest versions of Arduino / Teensyduino jus in case, but no change.


Here is the modified example code:

Code:
#include <USBHost_t36.h>

USBHost myusb;
MIDIDevice midi1(myusb);

void setup() {
  Serial.begin(115200);
  delay(1500);
  myusb.begin();

  midi1.setHandleNoteOn(myNoteOn);
  midi1.setHandleNoteOff(myNoteOff); 
  midi1.setHandleControlChange(myControlChange);
}

int NumNotes=0;

void loop() {
  myusb.Task();
  midi1.read();
}


void myNoteOn(byte channel, byte note, byte velocity) {
  Serial.print("Note On, ch=");
  Serial.print(channel, DEC);
  Serial.print(", note=");
  Serial.print(note, DEC);
  Serial.print(", velocity=");
  Serial.println(velocity, DEC);
  
  Serial.print(">>>>>>>> ");
  if(velocity>0)
    Serial.println(++NumNotes);
  else
    Serial.println(--NumNotes);  
}

void myNoteOff(byte channel, byte note, byte velocity) {
  Serial.print("Note Off, ch=");
  Serial.print(channel, DEC);
  Serial.print(", note=");
  Serial.print(note, DEC);
  Serial.print(", velocity=");
  Serial.println(velocity, DEC);
  
  Serial.print(">>>>>>>> ");
  Serial.println(--NumNotes);
}

And here is an example of a section of the the Serial output showing the problem for a missing note-on event:


Note On, ch=1, note=59, velocity=95
>>>>>>>> 1
Note Off, ch=1, note=59, velocity=0
>>>>>>>> 0
Note On, ch=1, note=62, velocity=125
>>>>>>>> 1
Note Off, ch=1, note=60, velocity=0
>>>>>>>> 0
Note Off, ch=1, note=62, velocity=0
>>>>>>>> -1


p.s. In this library example code, what is the purpose of myusb.Task() in the main loop? I can comment it out with no effect on the code's function, so I'm unclear on when this line would be needed and when it would not.
 
Last edited:
I tried your code with an M-Audio keyboard which has a modwheel. No matter how much I roll the wheel and play chords and arpeggios, the total note count is always zero at the end.
You didn't include your myControlChange function. I just copied this from an example:
Code:
void myControlChange(byte channel, byte control, byte value) {
  Serial.print("Control Change, ch=");
  Serial.print(channel, DEC);
  Serial.print(", control=");
  Serial.print(control, DEC);
  Serial.print(", value=");
  Serial.println(value, DEC);
}
Is your version any different?

Pete
 
I tried your code with an M-Audio keyboard which has a modwheel. No matter how much I roll the wheel and play chords and arpeggios, the total note count is always zero at the end.
You didn't include your myControlChange function. I just copied this from an example:
Code:
void myControlChange(byte channel, byte control, byte value) {
  Serial.print("Control Change, ch=");
  Serial.print(channel, DEC);
  Serial.print(", control=");
  Serial.print(control, DEC);
  Serial.print(", value=");
  Serial.println(value, DEC);
}
Is your version any different?

Pete

Thanks Pete,
Yes that's the same function, sorry I missed that. But even if I comment everything out in that function I can lose note events if changing controls very quickly. I've experimented some more and can get it to happen not just on the modwheel but also with other controls if I change them quickly enough whilst pressing down or releasing a key. The number of notes played is not so much the issue. I wonder if the m-audio sends changes more slowly. Certainly the small touch modwheel slider on the minilab can produce very fast changes. I do have to try harder to get it to lose note events by turning the rotary controls.
 
I've now tried two Teensy 3.6 boards, one USB powered, and the other with an external 5V supply, and get the same missing note events in both cases when I change controllers fast enough.
And no matter how hard I try, running my synth via the non-host USB midi port with the minilab II controller connected via midiOX doesn't lose any note events. So I don't think I have hardware issues.

Cheers,
Richard
 
I found an older Novation REMOTE SL37 keyboard controller and like Pete's M-Audio keyboard it behaves just fine too.

Maybe the Arturia minilab II, being a newer controller, is expecting midi USB host devices to be faster these days. Does anyone who knows the workings of the USBhost_t36 code think it plausible that it might not be fast enough for recent controllers?
 
I have not been able to make the Arturia minilab II work reliably with midi via the USB host port, so it seems the T36 host lib as it stands cannot be used in midi products that would be released to the general public to be used with contemporary keyboard controllers. I'm surprised there's not more interest in this?
 
I have not been able to make the Arturia minilab II work reliably with midi via the USB host port, so it seems the T36 host lib as it stands cannot be used in midi products that would be released to the general public to be used with contemporary keyboard controllers. I'm surprised there's not more interest in this?

@rvh: Regarding your earlier question on the use/need for calling myUSB.Task(), the USBhost source shows:

Code:
/**
 * \brief USB main task, responsible for enumeration and clean up stage.
 *
 * \note Must be periodically called from loop().
 */
void USBHost::Task(void)

You'd probably only notice a difference between including/excluding this call if/when connecting/disconnecting USB devices while running.

As for the main problem that you are observing, unfortunately, I don't have any direct suggestions to offer towards solving what you are experiencing. However, as an alternate data point, I can report the following: when exercising my TeensyMIDIPolySynth (using T4.1) using a MIDIPLUS AKM322 mini keyboard connected via the USBhost port, I am not able to reproduce any lost noteOn / noteOff messages with the simultaneous use of any other controls. This particular keyboard has a pitchbend wheel (reporting +8192 to -8192 counts), a modwheel (reporting 0 to 127 counts), a rotary encoder (reporting 0 to 127 counts), 3 digital pots (each reporting 0 to 127 counts), and a slider (reporting 0 to 127 counts). No matter how quickly I use any/all of these controls to generate CC messages while playing notes, I am not able to cause any message losses.

Because of the immense amount of source code (currently 33000+ LOC) for my TeensyMIDIPolySynth, I am compiling using "Smallest Code". You might try different optimization levels to see if that makes any difference in what you see. In addition, if you have a way to cool your T3.6, could you try over-clocking to see if that makes any difference ?? Have you checked your audio processing load ?? As a last resort, you might try a T4.x.

Good luck & have fun !!

Mark J Culross
KD5RXT
 
@rvh: Regarding your earlier question on the use/need for calling myUSB.Task(), the USBhost source shows:

Code:
/**
 * \brief USB main task, responsible for enumeration and clean up stage.
 *
 * \note Must be periodically called from loop().
 */
void USBHost::Task(void)

You'd probably only notice a difference between including/excluding this call if/when connecting/disconnecting USB devices while running.

As for the main problem that you are observing, unfortunately, I don't have any direct suggestions to offer towards solving what you are experiencing. However, as an alternate data point, I can report the following: when exercising my TeensyMIDIPolySynth (using T4.1) using a MIDIPLUS AKM322 mini keyboard connected via the USBhost port, I am not able to reproduce any lost noteOn / noteOff messages with the simultaneous use of any other controls. This particular keyboard has a pitchbend wheel (reporting +8192 to -8192 counts), a modwheel (reporting 0 to 127 counts), a rotary encoder (reporting 0 to 127 counts), 3 digital pots (each reporting 0 to 127 counts), and a slider (reporting 0 to 127 counts). No matter how quickly I use any/all of these controls to generate CC messages while playing notes, I am not able to cause any message losses.

Because of the immense amount of source code (currently 33000+ LOC) for my TeensyMIDIPolySynth, I am compiling using "Smallest Code". You might try different optimization levels to see if that makes any difference in what you see. In addition, if you have a way to cool your T3.6, could you try over-clocking to see if that makes any difference ?? Have you checked your audio processing load ?? As a last resort, you might try a T4.x.

Good luck & have fun !!

Mark J Culross
KD5RXT

Thanks Mark,

I very much appreciate any and all suggestions, as I'd really like to have the option of connecting controllers to my synth without a computer. But this is a synth I'd like to make into a product so (for now) I can't use the host midi port if it doesn't work with the Arturia Minilab II. It's a popular controller with a very small form-factor that pairs well synths because it has 16 rotary controllers as opposed to the more usual 8 or so.

I'm guessing it sends out midi data quicker than some other controllers being more recent, having more controllers, and two small touch sliders for pitch bend and modwheel that allow you to go from one extreme to the other very quickly. The fact that the Arturia works fine with the regular usb midi port on the teensy 3.6 points to the t36 host lib code being the bottle neck, but I'm hoping someone who knows its workings can confirm that's plausible (and ideally fixable). I've also looked at the output from the Arturia with midi OX and see nothing indicating the Arturia is doing anything it shouldn't be.

If you look at the example code above, it's a completely minimal t36 USB host library example that only prints incoming midi note events. There's no audio or anything else running, yet it easily loses midi note on/off events using the Arturia but not my older controllers, and even when there is no code being executed in the midi CC callback.

If I get a chance I may check if that example on a Teensy 4.1 has the same problem, but unfortunately I can't use the 4.1 for my application because I need the integrated DAC from the the 3.6 for its ability to play audio samples at any time rather than at a steady sample rate.

Cheers,
Richard
 
Last edited:
I tried overclocking the T36 at 240Mhz (rather than 180Mhz) but it didn't solve my problem.
 
Thanks Mark,

I very much appreciate any and all suggestions, as I'd really like to have the option of connecting controllers to my synth without a computer. But this is a synth I'd like to make into a product so (for now) I can't use the host midi port if it doesn't work with the Arturia Minilab II. It's a popular controller with a very small form-factor that pairs well synths because it has 16 rotary controllers as opposed to the more usual 8 or so.

I'm guessing it sends out midi data quicker than some other controllers being more recent, having more controllers, and two small touch sliders for pitch bend and modwheel that allow you to go from one extreme to the other very quickly. The fact that the Arturia works fine with the regular usb midi port on the teensy 3.6 points to the t36 host lib code being the bottle neck, but I'm hoping someone who knows its workings can confirm that's plausible (and ideally fixable). I've also looked at the output from the Arturia with midi OX and see nothing indicating the Arturia is doing anything it shouldn't be.

Cheers,
Richard

Richard:

I took another look at my TeensyMIDIPolySynth source & realized that there is one additional difference in my implementation of the USBHost interface that might be pertinent. Because I have only used T4.x for all of my projects, I don't know if this is even valid and/or would make any difference, but could you try the following when declaring your USBHost interface:

Code:
USBHost myusb;
MIDIDevice_BigBuffer midi1(myusb);     // [COLOR="#00FF00"]<<<=====[/COLOR]

In searching the forum for more clues, it's easy to find other reports of occasional MIDI data loss & the common answer is to use the "MIDIDevice_BigBuffer" declaration (which is probably how I ended up with that in my source). See <this> example post & <this> post, both by Paul, as just two examples w/ explanation of the difference.

Hope that helps . . .

Mark J Culross
KD5RXT
 
Richard:

I took another look at my TeensyMIDIPolySynth source & realized that there is one additional difference in my implementation of the USBHost interface that might be pertinent. Because I have only used T4.x for all of my projects, I don't know if this is even valid and/or would make any difference, but could you try the following when declaring your USBHost interface:

Code:
USBHost myusb;
MIDIDevice_BigBuffer midi1(myusb);     // [COLOR="#00FF00"]<<<=====[/COLOR]

In searching the forum for more clues, it's easy to find other reports of occasional MIDI data loss & the common answer is to use the "MIDIDevice_BigBuffer" declaration (which is probably how I ended up with that in my source). See <this> example post & <this> post, both by Paul, as just two examples w/ explanation of the difference.

Hope that helps . . .

Mark J Culross
KD5RXT

Thanks Mark, yes I had seen those threads and have tried that. Unfortunately it makes no difference with my issue.
 
p.s. if anyone else happens to have an Arturia Minilab II and is willing to try the above example code whilst playing notes and sweeping the modwheel slider it would be good to get verification of the problem.

Or if you don't have one, but could do with a small 2-octave 16-knob controller for just US$99... :)
 
I just tried the simple example on a Teensy 4.0 and get the same result as with the T3.6, with missing note events if I simultaneously vary the modwheel touchpad rapidly on the minilab.

So perhaps it's not a processing speed issue per se, but some kind of queuing/buffering problem despite including the _BigBuffer option?

Here is the complete code again including the function I missed last time, and including the BigBuffer option that uses the higher bus speed:

Code:
#include <USBHost_t36.h>

USBHost myusb;
//MIDIDevice midi1(myusb);
MIDIDevice_BigBuffer midi1(myusb); 

void setup() {
  Serial.begin(115200);
  delay(1500);
  myusb.begin();

  midi1.setHandleNoteOn(myNoteOn);
  midi1.setHandleNoteOff(myNoteOff); 
  midi1.setHandleControlChange(myControlChange);
}

int NumNotes=0;

void loop() {
  myusb.Task();
  midi1.read();
}


void myNoteOn(byte channel, byte note, byte velocity) {
  Serial.print("Note On, ch=");
  Serial.print(channel, DEC);
  Serial.print(", note=");
  Serial.print(note, DEC);
  Serial.print(", velocity=");
  Serial.println(velocity, DEC);
  
  Serial.print(">>>>>>>> ");
  if(velocity>0)
    Serial.println(++NumNotes);
  else
    Serial.println(--NumNotes);  
}

void myNoteOff(byte channel, byte note, byte velocity) {
  Serial.print("Note Off, ch=");
  Serial.print(channel, DEC);
  Serial.print(", note=");
  Serial.print(note, DEC);
  Serial.print(", velocity=");
  Serial.println(velocity, DEC);  
  Serial.print(">>>>>>>> ");
  Serial.println(--NumNotes);
}


void myControlChange(byte channel, byte control, byte value) {

}
 
Responding to PM: Sorry know very little about using MIDI and do not have any test setups... So not sure how much help I could be.

About the only way I see that I could potentially take a look, is if there is some specific setup, that reproduces the issue.

For example if you somehow capture the midi information that is going to the T3.6 or T4.x over the USBHost and for example program another Teensy to generate that midi stream, that does not require any additional hardware, that you could plug into the USB and reproduce the issue with simple program like you show in the previous post.
Along with details on what you are seeing and what expected.

Now if it were me, and were trying to debug it...
1) My guess is this runs at 480mbs so LA can not capture the USB traffic... So that is probably out. And I don't have USB analyzer... So would need a different approach.
2) I would try to capture, what information the USBHost is seeing and see if it actually sees the data... Again not sure what that data is
My guess is printing all of this to USB Serial may change timing and make it work differently. SO I would probably capture the information to memory...
Would probably try on T4.1 with PSRAM,

In some cases I have hacked up a memory stream to handle this... SOmething like:
Code:
// Quick and Dirty memory Stream...
class RAMStream : public Stream {
public:
  // overrides for Stream
  virtual int available() { return (tail_ - head_); }
  virtual int read() { return (tail_ != head_) ? buffer_[head_++] : -1; }
  virtual int peek() { return (tail_ != head_) ? buffer_[head_] : -1; }

  // overrides for Print
  virtual size_t write(uint8_t b) {
    if (tail_ < sizeof(buffer_)) {
      buffer_[tail_++] = b;
      return 1;
    }
    return 0;
  }

  enum { BUFFER_SIZE = 32768 };
  uint8_t buffer_[BUFFER_SIZE];
  uint32_t head_ = 0;
  uint32_t tail_ = 0;
};
Again if I thought I had tons of data that might be received, I would make the storage in PSRAM... and maybe up to a couple Megabytes...

Then do all of the debug output to the stream...

And when you detect the event, then have some way to then tell this stream to print out, in an example sketch that I used the above in.
I was doing this as Serial was not valid yet (initializing MTP), so then after Serial was valid, I did something like:
Code:
  int ch;
  while ((ch = rstream.read()) != -1)
    Serial.write(ch);
Obviously could be done more efficiently, but gets the job done.

But the hard part is figuring out what data to capture... Also if this generates 1000s of lines of output, may want to make stream code smarter, and only dump out a range of
the data...

But again sorry, without knowing a lot more about MIDI and your setup, not sure anything specific I can say to help
 
Thanks Kurt,

I'll have a think about how to capture a stream that produces the host port data loss so it can be reproduced without the midi controller.

Cheers,
Richard
 
Now if it were me, and were trying to debug it...
1) My guess is this runs at 480mbs so LA can not capture the USB traffic... So that is probably out. And I don't have USB analyzer... So would need a different approach.
2) I would try to capture, what information the USBHost is seeing and see if it actually sees the data... Again not sure what that data is
My guess is printing all of this to USB Serial may change timing and make it work differently.

I finally had some time to return to this issue in the hope of being able to have a robust host midi-port on my synth. I'm running all this on T4.1 now, and seeing the same behaviour as with the T3.6.

I'm not clear how I can capture the actual (host) midi data from the midi keyboard so you can recreate the problem, because the note events are already missing when retrieved by the host USB input callback routines that I would use to record the data.

As an alternative, are there any variable values in the host USB core code that I could try changing to see if it helps?
 
Last edited:

Seems like a different issue to me; host midi out rather than midi in for one thing. And I only get the missing note events when I simultaneously change the modwheel quickly while also playing keys/notes on this particular midi keyboard (Arturua minilab II). I'm assuming it may be because the modwheel is a touch slider that can produce much faster modwheel events than a mechanical modwheel that you would find on most keyboards.

At first I thought it might actually be a faulty touch slider, but it works fine if I use the midi keybaord via a computer and then into the non-host usb midi input on the teensy. Also puzzling to me is that it seems only with the modwheel. I can turn pitch wheel and other control knobs as much as I like without adverse consequences.
 
2) I would try to capture, what information the USBHost is seeing and see if it actually sees the data... Again not sure what that data is
My guess is printing all of this to USB Serial may change timing and make it work differently.
...
But again sorry, without knowing a lot more about MIDI and your setup, not sure anything specific I can say to help

At the risk of over-explaining, here are a bunch more details in case you haven't looked at midi at all. The setup when running the code from #13 is a small minilab II midi controller keyboard connected to the T4.1 host USB port. It gets both power and data from the teensy, it has a touch 'modwheel' slider and two octave music keyboard, but produces no sound. The teensy is also connected to the computer for serial messages. Any modwheel position change causes the controller to send out a short 3-byte midi message (0xB0, 0x01, wheel value) that can be used by a synth to modulate parameters. Pressing a music-keyboard key produces a 3-byte midi note-on sequence (0x90, note, velocity) and similarly 3 bytes for a note off (0x80,note,vel). So for every note-on there should be a corresponding note-off event when all keys are released.

I run the test program above, and for a few seconds move the minilab modwheel slider up and down quickly while also playing a few notes on the keyboard. When I stop playing, the number of notes still on, as reported via the serial port, should be 0. If I move the modwheel quickly enough during the test, even just a few notes is enough to get a missing note event and therefore a non-0 result at the end. When using the host midi port in this way in my full synth code, this often results in a 'hanging' notes that keep sounding when all keys have been released.

If I put serial print messages describing the note on/off events in my synth code, they correspond exactly to what I hear in terms of hanging notes. Adding the USB Serial print doesn't seem to change any salient timing because the hanging note behaviour is unaffected.

Running the same tests via the non-host USB midi port doesn't produce hanging notes, no matter how fast I change the nodwheel. So it seems that the host callback simply isn't receiving some of the midi note events when I send (fast) modwheel vlaues. It may be the case that modwheel events themselves are also missing, but that's much harder to discern.

In these tests, were not necessarily talking about large amounts of modwheel data, just fast changes I believe. I can get hanging notes in test runs that include just a few dozen modwheel messages.
 
@rvh: Have an Arturia MiniLab MkII here and a Teensy 4.1 so I could do some testing the coming days.
Is your latest code, the code that you posted in message #13?

Paul
 
I can confirm the issue:

Untitled.png

Using the code from message 13.

Paul
 
I can confirm the issue:
...
Using the code from message 13.

Paul

Thanks very much Paul. It's good to know there's nothing unusual bout my particular minilab. Unfortunately it means other midi devices may also not work reliably using the current version of hostUSB_t36. I'm hopeful the author(s) of that code may be able to take a look at it, and I would be willing to arrange a minilab mk II for that purpose if necessary. :)
 
The only hub I have atm is a small unpowered one, which doesn't work at all when inserted between the T4.1 and the midi controller.
 
@rvh & @PaulS:

Unfortunately, I don't have an Arturia MiniLab MKII here, so can't provide any local testing of this proposal for you. Can you try the following modified (mods in RED) sketch to see if you see any positive effect (you'll need to make sure to select the USB Type in the Arduino IDE as "Serial + MIDI"):

Code:
#include <USBHost_t36.h>
[COLOR="#FF0000"]#include <MIDI.h>
[/COLOR]
[COLOR="#FF0000"]USBHost myusb;
USBHub hub1(myusb);
MIDIDevice_BigBuffer midi1(myusb);

MIDI_CREATE_DEFAULT_INSTANCE();
[/COLOR]

void setup() {
  Serial.begin(115200);
  delay(1500);

[COLOR="#FF0000"]  midi1.setHandleNoteOn(myNoteOn);
  midi1.setHandleNoteOff(myNoteOff); 
  midi1.setHandleControlChange(myControlChange);

  MIDI.begin(MIDI_CHANNEL_OMNI);
  myusb.begin();
[/COLOR]}

int NumNotes=0;

void loop() {
[COLOR="#FF0000"]   MIDI.read();
   usbMIDI.read();
   myusb.Task();
   midi1.read();
[/COLOR]}


void myNoteOn(byte channel, byte note, byte velocity) {
  Serial.print("Note On, ch=");
  Serial.print(channel, DEC);
  Serial.print(", note=");
  Serial.print(note, DEC);
  Serial.print(", velocity=");
  Serial.println(velocity, DEC);
  
  Serial.print(">>>>>>>> ");
  if(velocity>0)
    Serial.println(++NumNotes);
  else
    Serial.println(--NumNotes);  
}

void myNoteOff(byte channel, byte note, byte velocity) {
  Serial.print("Note Off, ch=");
  Serial.print(channel, DEC);
  Serial.print(", note=");
  Serial.print(note, DEC);
  Serial.print(", velocity=");
  Serial.println(velocity, DEC);  
  Serial.print(">>>>>>>> ");
  Serial.println(--NumNotes);
}


void myControlChange(byte channel, byte control, byte value) {

}
 
Last edited:
Back
Top