MIDI using USBHost_T36 library slow/lagging.

Status
Not open for further replies.

sixeight

Well-known member
I have developed a MIDI foot controller that uses a Teensy 3.6 and has midi connections via USB host, USB regular and via serial midi (2x). I noticed that sending MIDI messages out from my foot controller that the USB Host port midi is lagging. When I connect the same device via the other midi ports, there is no lagging.

The lagging is a problem when i operate an expression pedal. The device I control is responding laggy, making the use of expression pedals useless, when connecting through the USB host port. Other ports work fine. But using other ports means bringing extra gear (external computer) which I do not want to do.

Here is how the host library is called:
Code:
#include <USBHost_t36.h>

USBHost myusb;
USBHub hub1(myusb);
USBHub hub2(myusb);
USBHub hub3(myusb);
USBHub hub4(myusb);
#define NUMBER_OF_USB_HOST_MIDI_PORTS 8
MIDIDevice_BigBuffer usbhMIDI01(myusb); // With bigbuffer, newer Roland devices are working properly!
MIDIDevice_BigBuffer usbhMIDI02(myusb);
MIDIDevice_BigBuffer usbhMIDI03(myusb);
MIDIDevice_BigBuffer usbhMIDI04(myusb);
MIDIDevice_BigBuffer usbhMIDI05(myusb);
MIDIDevice_BigBuffer usbhMIDI06(myusb);
MIDIDevice_BigBuffer usbhMIDI07(myusb);
MIDIDevice_BigBuffer usbhMIDI08(myusb);
MIDIDevice_BigBuffer * usbhMIDI[NUMBER_OF_USB_HOST_MIDI_PORTS] = {
  &usbhMIDI01, &usbhMIDI02, &usbhMIDI03, &usbhMIDI04, &usbhMIDI05, &usbhMIDI06, &usbhMIDI07, &usbhMIDI08
};

In loop I have included:
Code:
myusb.Task();
  for (uint8_t p = 0; p < NUMBER_OF_USB_HOST_MIDI_PORTS; p++) {
    VCbridge_in_port[USBHMIDI_PORT >> 4] = p + 1;
    usbhMIDI[p]->read();
  }

And sending the actual midi messages:
Code:
usbhMIDI[Port]->sendSysEx(sxlength, sxdata, true, cable); // hasterm = true, which means no extra 0xF0 and 0xF7 bytes will be added

Looking at the source code of the USBHost_t36 library, whenever a midi message is sent, it is queued. Adding extra myUSB.Task() commands does not seem to help. is there anything I can do to speed up USB? I tried removing a number of instances of USBHub and MIDIDevice_BigBuffer, but that does not seem to make any difference in processing speed.
 
Did some further tests. Other MIDI devices connected to the USB host port also respond slow to midi messages sent via this library. Is there any way to speed up USB Host communication, or is this not possible. Would a Teensy 4.0 with its faster clock speed give better results?
 
Sorry, I have done a reasonable amount of work in this library, but I have never touched the MIDI stuff. Mainly because I have never done anything with MIDI...

For example I have no clue if something like: usbhMIDI...->send_now();
Might work in a similar way to things like Serial ports, where you might do: Serial1.flush();
Which says send out what you have now.

Assuming something like that works, you can often decrease latency issue, but it may also decrease throughput. Again not sure if that is an issue for you or not...
 
Thanks KurtE for responding. I have tried the send_now() command, but it does not seem to make any difference. So maybe it is not a timing issue, but something else.

I have tested the library with a different MIDI device. That device is showing the error MIDI buffer full. I do not get these errors when I connect through a Raspberry Pi (using the ALSA MIDI driver system). So maybe data is being sent several times or some byte in the data stream is not right. Both MIDI devices are Roland devices.
 
Now I am in no way in expert in USB Midi. But I am comparing the USBHost_t36 library with https://github.com/YuuichiAkagawa/USBH_MIDI/blob/master/usbh_midi.cpp, the MIDI library that goes with the Arduino host shield.

Could it be that MIDI sysex messages in the t36 library are split into several separate USB messages, where the Arduino midi host library sends it as one package?

In USBHost_t36/midi.cpp I read:
Code:
void MIDIDeviceBase::write_packed(uint32_t data)
{
	if (!txpipe) return;
	uint32_t tx_max = tx_size / 4;
	while (1) {
		uint32_t tx1 = tx1_count;
		uint32_t tx2 = tx2_count;
		if (tx1 < tx_max && (tx2 == 0 || tx2 >= tx_max)) {
			// use tx_buffer1
			tx_buffer1[tx1++] = data;
			tx1_count = tx1;
			if (tx1 >= tx_max) {
				queue_Data_Transfer(txpipe, tx_buffer1, tx_max*4, this);
			} else {
				// TODO: start a timer, rather than sending the buffer
				// before it's full, to make best use of bandwidth
				tx1_count = tx_max;
				queue_Data_Transfer(txpipe, tx_buffer1, tx_max*4, this);
			}
			return;
		}
		if (tx2 < tx_max) {
			// use tx_buffer2
			tx_buffer2[tx2++] = data;
			tx2_count = tx2;
			if (tx2 >= tx_max) {
				queue_Data_Transfer(txpipe, tx_buffer2, tx_max*4, this);
			} else {
				// TODO: start a timer, rather than sending the buffer
				// before it's full, to make best use of bandwidth
				tx2_count = tx_max;
				queue_Data_Transfer(txpipe, tx_buffer2, tx_max*4, this);
			}
			return;
		}
	}
}

void MIDIDeviceBase::send_sysex_buffer_has_term(const uint8_t *data, uint32_t length, uint8_t cable)
{
	cable = (cable & 0x0F) << 4;
    
	while (length > 3) {
		write_packed(0x04 | cable | (data[0] << 8) | (data[1] << 16) | (data[2] << 24));
		data += 3;
		length -= 3;
	}
	if (length == 3) {
		write_packed(0x07 | cable | (data[0] << 8) | (data[1] << 16) | (data[2] << 24));
	} else if (length == 2) {
		write_packed(0x06 | cable | (data[0] << 8) | (data[1] << 16));
	} else if (length == 1) {
		write_packed(0x05 | cable | (data[0] << 8));
	}
}

In https://github.com/YuuichiAkagawa/USBH_MIDI/blob/master/usbh_midi.cpp I read:
Code:
/* Send SysEx message to MIDI device */
uint8_t USBH_MIDI::SendSysEx(uint8_t *dataptr, uint16_t datasize, uint8_t nCable)
{
        uint8_t buf[MIDI_EVENT_PACKET_SIZE];
        uint8_t rc = 0;
        uint16_t n = datasize;
        uint16_t pktSize = (n*10/3+7)/10*4;   //Calculate total USB MIDI packet size
        uint8_t wptr = 0;
        uint8_t maxpkt = epInfo[epDataInIndex].maxPktSize;

        if( maxpkt > MIDI_EVENT_PACKET_SIZE ) maxpkt = MIDI_EVENT_PACKET_SIZE;

        USBTRACE("SendSysEx:\r\t");
        USBTRACE2(" Length:\t", datasize);
        USBTRACE2(" Total pktSize:\t", pktSize);

        while(n > 0) {
                //Byte 0
                buf[wptr] = (nCable << 4) | 0x4;             //x4 SysEx starts or continues

                switch ( n ) {
                    case 1 :
                        buf[wptr++] = (nCable << 4) | 0x5;   //x5 SysEx ends with following single byte.
                        buf[wptr++] = *(dataptr++);
                        buf[wptr++] = 0x00;
                        buf[wptr++] = 0x00;
                        n = n - 1;
                        break;
                    case 2 :
                        buf[wptr++] = (nCable << 4) | 0x6;   //x6 SysEx ends with following two bytes.
                        buf[wptr++] = *(dataptr++);
                        buf[wptr++] = *(dataptr++);
                        buf[wptr++] = 0x00;
                        n = n - 2;
                        break;
                    case 3 :
                        buf[wptr]   = (nCable << 4) | 0x7;   //x7 SysEx ends with following three bytes.
                    default :
                        wptr++;
                        buf[wptr++] = *(dataptr++);
                        buf[wptr++] = *(dataptr++);
                        buf[wptr++] = *(dataptr++);
                        n = n - 3;
                        break;
                }

                if( wptr >= maxpkt || n == 0 ){ //Reach a maxPktSize or data end.
                        USBTRACE2(" wptr:\t", wptr);
                        if( (rc = pUsb->outTransfer(bAddress, epInfo[epDataOutIndex].epAddr, wptr, buf)) != 0 ){
                                break;
                        }
                        wptr = 0;  //rewind data pointer
                }
        }
        return(rc);
}
 
I found a solution to the slowness of the USBmidi with Roland devices. My hunch in the previous post was correct. The USBHost library only sends one 32 bit message in every USB package. But Roland supports multiple MIDI messages in one package. Not sure if normal USB MIDI does this as well. Multiple sysex USB packets are not working for a ZOOM MIDI device, which is class compliant. So I added a workaround for that.

Maybe somebody with more knowledge of USB midi can chime in.

Would it be possible to add this solution to the USBHost_t36 library?

Here is the solution:

In USBHost_t36 I added the following lines:
Code:
protected:
    void add_sysex_packed(uint32_t data);
    void write_sysex_message();

private:
    uint32_t msg_sysex_packed[SYSEX_MAX_LEN / 3];
    uint16_t msg_sysex_len_packed;
    bool support_sysex_multi_message;

In midi.cpp the following procedures were changed or altered:
Code:
bool MIDIDeviceBase::claim(Device_t *dev, int type, const uint8_t *descriptors, uint32_t len)
{
	// only claim at interface level
	if (type != 1) return false;
	println("MIDIDevice claim this=", (uint32_t)this, HEX);
	println("len = ", len);

	const uint8_t *p = descriptors;
	const uint8_t *end = p + len;

	if (p[0] != 9 || p[1] != 4) return false; // interface descriptor
	//println("  bInterfaceClass=", p[5]);
	//println("  bInterfaceSubClass=", p[6]);
	bool ismidi = false;
	if (p[5] == 1 && p[6] == 3) {
		println("  Interface is MIDI"); // p[5] is bInterfaceClass: 1 = Audio class
		ismidi = true;                  // p[6] is bInterfaceSubClass: 3 = MIDI
	} else {
		if (p[5] >= 2 && p[5] <= 18) return false; // definitely not MIDI
		// Yamaha uses vendor specific class, but can be
		// identified as MIDI from CS_INTERFACE descriptors.
		//  https://forum.pjrc.com/threads/55142?p=199162&viewfull=1#post199162
		println("  Interface is unknown (might be Yahama)");
	}
	p += 9;
	rx_ep = 0;
	tx_ep = 0;

	while (p < end) {
		len = *p;
		if (len < 4) return false; // all audio desc are at least 4 bytes
		if (p + len > end) return false; // reject if beyond end of data
		uint32_t type = p[1];
		print("type: ", type);
		println(", len: ", len);
		if (type == 4 || type == 11) break; // interface or IAD, not for us
		if (type == 0x24) {  // 0x24 = Audio CS_INTERFACE, audio 1.0, page 99
			uint32_t subtype = p[2];
			//println("subtype: ", subtype);
			if (subtype == 1) {
				// Interface Header, midi 1.0, page 21
				println("    MIDI Header (ignored)");
				ismidi = true;
                support_sysex_multi_message = false;
			} else if (subtype == 2) {
				// MIDI IN Jack, midi 1.0, page 22
				println("    MIDI IN Jack (ignored)");
				ismidi = true;
                support_sysex_multi_message = false;
			} else if (subtype == 3) {
				// MIDI OUT Jack, midi 1.0, page 22
				println("    MIDI OUT Jack (ignored)");
				ismidi = true;
                support_sysex_multi_message = false;
			} else if (subtype == 4) {
				// Element Descriptor, midi 1.0, page 23-24
				println("    MIDI Element (ignored)");
				ismidi = true;
                support_sysex_multi_message = false;
			} else if (subtype == 0xF1 && p[3] == 2) {
				// see Linux sound/usb/quirks.c create_roland_midi_quirk()
				println("    Roland vendor-specific (ignored)");
				ismidi = true;
                support_sysex_multi_message = true;
			} else {
				println("    Unknown MIDI CS_INTERFACE descriptor!");
				return false; // unknown
			}
		} else if (type == 5) {
			// endpoint descriptor
			if (p[0] < 7) return false; // at least 7 bytes
			if (p[3] != 2 && p[3] != 3) return false; // must be bulk or interrupt type
			println("    MIDI Endpoint: ", p[2], HEX);
			switch (p[2] & 0xF0) {
			case 0x80:
				// IN endpoint
				if (rx_ep == 0) {
					rx_ep = p[2] & 0x0F;
					rx_ep_type = p[3];
					rx_size = p[4] | (p[5] << 8);
					println("      rx_size = ", rx_size);
				}
				break;
			case 0x00:
				// OUT endpoint
				if (tx_ep == 0) {
					tx_ep = p[2];
					tx_ep_type = p[3];
					tx_size = p[4] | (p[5] << 8);
					println("      tx_size = ", tx_size);
				}
				break;
			default:
				return false;
			}
		} else if (type == 37) {
			// MIDI endpoint info, midi 1.0: 6.2.2, page 26
			println("    MIDI Endpoint Jack Association (ignored)");
		} else {
			println("    Unknown descriptor, type=", type);
			return false; // unknown
		}
		p += len;
	}
	if (!ismidi) {
		println("This interface is not MIDI");
		return false;
	}
	// if an IN endpoint was found, create its pipe
	if (rx_ep && rx_size <= max_packet_size) {
		rxpipe = new_Pipe(dev, rx_ep_type, rx_ep, 1, rx_size);
		if (rxpipe) {
			rxpipe->callback_function = rx_callback;
			queue_Data_Transfer(rxpipe, rx_buffer, rx_size, this);
			rx_packet_queued = true;
		}
	} else {
		rxpipe = NULL;
	}
	// if an OUT endpoint was found, create its pipe
	if (tx_ep && tx_size <= max_packet_size) {
		txpipe = new_Pipe(dev, tx_ep_type, tx_ep, 0, tx_size);
		if (txpipe) {
			txpipe->callback_function = tx_callback;
			tx1_count = 0;
			tx2_count = 0;
		}
	} else {
		txpipe = NULL;
	}
	rx_head = 0;
	rx_tail = 0;
	msg_channel = 0;
	msg_type = 0;
	msg_data1 = 0;
	msg_data2 = 0;
	msg_sysex_len = 0;
	// claim if either pipe created
	return (rxpipe || txpipe);
}

void MIDIDeviceBase::add_sysex_packed(uint32_t data){
    if (!support_sysex_multi_message) { // Just send the message...
        write_packed(data);
        return;
    }
    msg_sysex_packed[msg_sysex_len_packed++] = data;
    if (msg_sysex_len_packed >= (SYSEX_MAX_LEN / 3)) write_sysex_message();
}

void MIDIDeviceBase::write_sysex_message() {
    if (!txpipe) return;
    if (!support_sysex_multi_message) return;
    uint32_t tx_max = tx_size / 4;
    while (1) {
        uint32_t tx1 = tx1_count;
        uint32_t tx2 = tx2_count;
        if (tx1 + msg_sysex_len_packed <= tx_max && (tx2 == 0 || tx2 >= tx_max)) {
            // use tx_buffer1
            for (uint16_t i = 0; i < msg_sysex_len_packed; i++) {
                tx_buffer1[tx1++] = msg_sysex_packed[i];
                if (tx1 >= tx_max) {
                    queue_Data_Transfer(txpipe, tx_buffer1, tx_max*4, this);
                }
            }
            tx1_count = tx1;
            if (tx1 >= tx_max) {
                queue_Data_Transfer(txpipe, tx_buffer1, tx_max*4, this);
            } else {
                // TODO: start a timer, rather than sending the buffer
                // before it's full, to make best use of bandwidth
                tx1_count = tx_max;
                queue_Data_Transfer(txpipe, tx_buffer1, tx_max*4, this);
            }
            msg_sysex_len_packed = 0;
            return;
        }
        if (tx2 + msg_sysex_len_packed <= tx_max) {
            // use tx_buffer2
            for (uint16_t i = 0; i < msg_sysex_len_packed; i++) {
                tx_buffer2[tx2++] = msg_sysex_packed[i];
                if (tx2 >= tx_max) {
                    queue_Data_Transfer(txpipe, tx_buffer2, tx_max*4, this);
                }
            }
            tx2_count = tx2;
            if (tx2 >= tx_max) {
                queue_Data_Transfer(txpipe, tx_buffer2, tx_max*4, this);
            } else {
                // TODO: start a timer, rather than sending the buffer
                // before it's full, to make best use of bandwidth
                tx2_count = tx_max;
                queue_Data_Transfer(txpipe, tx_buffer2, tx_max*4, this);
            }
            msg_sysex_len_packed = 0;
            return;
        }
    }
}

void MIDIDeviceBase::send_sysex_buffer_has_term(const uint8_t *data, uint32_t length, uint8_t cable)
{
	cable = (cable & 0x0F) << 4;
    msg_sysex_len_packed = 0;
	while (length > 3) {
		add_sysex_packed(0x04 | cable | (data[0] << 8) | (data[1] << 16) | (data[2] << 24));
		data += 3;
		length -= 3;
	}
	if (length == 3) {
		add_sysex_packed(0x07 | cable | (data[0] << 8) | (data[1] << 16) | (data[2] << 24));
        write_sysex_message();
	} else if (length == 2) {
		add_sysex_packed(0x06 | cable | (data[0] << 8) | (data[1] << 16));
        write_sysex_message();
	} else if (length == 1) {
		add_sysex_packed(0x05 | cable | (data[0] << 8));
        write_sysex_message();
	}
}

void MIDIDeviceBase::send_sysex_add_term_bytes(const uint8_t *data, uint32_t length, uint8_t cable)
{
	cable = (cable & 0x0F) << 4;
    msg_sysex_len_packed = 0;

	if (length == 0) {
		add_sysex_packed(0x06 | cable | (0xF0 << 8) | (0xF7 << 16));
        write_sysex_message();
		return;
	} else if (length == 1) {
		add_sysex_packed(0x07 | cable | (0xF0 << 8) | (data[0] << 16) | (0xF7 << 24));
        write_sysex_message();
		return;
	} else {
		add_sysex_packed(0x04 | cable | (0xF0 << 8) | (data[0] << 16) | (data[1] << 24));
		data += 2;
		length -= 2;
	}
	while (length >= 3) {
		add_sysex_packed(0x04 | cable | (data[0] << 8) | (data[1] << 16) | (data[2] << 24));
		data += 3;
		length -= 3;
	}
	if (length == 2) {
		add_sysex_packed(0x07 | cable | (data[0] << 8) | (data[1] << 16) | (0xF7 << 24));
        write_sysex_message();
	} else if (length == 1) {
		add_sysex_packed(0x06 | cable | (data[0] << 8) | (0xF7 << 16));
        write_sysex_message();
	} else {
		add_sysex_packed(0x05 | cable | (0xF7 << 8));
        write_sysex_message();
	}
}
 
I found out normal MIDI also supports multiple messages per USB packet. I have done some further experiments with my changes in the previous post. But I found the Teensy hangs whenever I send a message that is larger than the buffer. It must be something to do with the dual tx buffers. I probably do not switch between the buffers properly. Really do not understand what I am doing here or how to fix it.

All the code is in the previous post. I also forked the full library here: https://github.com/sixeight7/USBHost_t36

Anybody can help?
 
Reviving an old thread. I have developed the MIDI usb host library further. It now supports multiple messages per USB packet for regular MIDI too.
The updated code is in my Github repository: https://github.com/sixeight7/USBHost_t36

But I have one specific midi device that only works without a hub. As soon as the device is connected through a hub, I found the library is hanging, because the buffers overflow. Other midi devices work fine through this same hub. Also tried other hubs, but the same thing is happening. The buffers overflow: tx1_count and tx2_count both get to the tx_max value and are never cleared. This probably means that the tx_callback function is not called. But I cannot find the line number in the library where the tx_callback function is called. Who can point me in the right direction? How can I debug this further?
 
I just registered to say thank you for such a great fork :eek: . I am using a Launchpad Mini Mk3 as a simple game display/controller, and I ran into issues wanting to have tighter timings. Thanks to your modifications, sending my SysEx message went down from ~70ms to just 4ms. I really hope these changes make their way into the official library.

PS. I really hope this kind of posts are not against the forums rules, since I was not able to find them.
 
Status
Not open for further replies.
Back
Top