MIDI Data Thinning/Running Status, etc.

Status
Not open for further replies.
F

ForestCat

Member
Without going crazy w/ details, I have an arduino app which monitors incoming MIDI data from a number of h/w MIDI controllers, and retransmits CC's on 6 different channels ( to six connected h/w synths). I've been trying to troubleshoot a maddening problem, in which it works well with some input h/w(nearly instantaneous & accurate echo/re-channelization of the cc's on all six channels), while other input h/w suffers from EXTREME lag, lost data, etc.

I think I'm stating to understand why. After doing arduino serial captures of the input data from the various h/w, it looks like the devices that work well utilize running status, and/or transmit way less data when the controller (Expression pedal, etc) is moved QUICKLY.

The most problematic input device is my home-brewed Teensy which has an assortment of softpots, Force Sensing Resistors, etc, as CC generators.

is there established method for "spacing out" the data bytes, i.e. 1,4,7,10,13,16, etc., when a control is swept through it's range quickly? or a way to implement running status?

At my wits end, any insight greatly appreciated...
 
ForestCat said:
Without going crazy w/ details, I have an arduino app which monitors incoming MIDI data from a number of h/w MIDI controllers, and retransmits CC's on 6 different channels ( to six connected h/w synths). I've been trying to troubleshoot a maddening problem, in which it works well with some input h/w(nearly instantaneous & accurate echo/re-channelization of the cc's on all six channels), while other input h/w suffers from EXTREME lag, lost data, etc.

I think I'm stating to understand why. After doing arduino serial captures of the input data from the various h/w, it looks like the devices that work well utilize running status, and/or transmit way less data when the controller (Expression pedal, etc) is moved QUICKLY.

The most problematic input device is my home-brewed Teensy which has an assortment of softpots, Force Sensing Resistors, etc, as CC generators.

is there established method for "spacing out" the data bytes, i.e. 1,4,7,10,13,16, etc., when a control is swept through it's range quickly? or a way to implement running status?

At my wits end, any insight greatly appreciated...
Click to expand...

I assume you are sending the MIDI out of the teensy using a 5-Pin MIDI Din port? What version of the MIDI library are you using? The easiest way to reduce the amount of data is to simply measure less often.

Code: 
elapsedMillis elapsed;

if (elapsed > 5) {
measure();
elapsed = 0;
}
 
ForestCat said:
is there established method for "spacing out" the data bytes, i.e. 1,4,7,10,13,16, etc., when a control is swept through it's range quickly?
Click to expand...

Yes. Many ways, actually, all of which involve adding code to check millis() or use elapsedMillis variables. The details really depend on the structure of your project's code, which we can't see, so this very generic advice is the best I can do.

or a way to implement running status?
Click to expand...

Yes, of course there's a way. The MIDI library already does this. But of course we can't see whether you're using that lib or rolling your own with writes to Serial1, or using some other way.

At my wits end, any insight greatly appreciated...
Click to expand...

We have the "Forum Rule" in red at the top of every page, specifically so these sorts of questions can be answered well, with useful advice that pertains to exactly your needs. But if you don't follow the Forum Rule, nobody can see the details, so the best help you can get is this sort of very generic answer.
 
I would make a buffer to put the message_data bytes, indexed by (message_byte & 0xEF) where you only have a single entry for any given message, and just limit how often you transmit.

You would need an additional buffer of 16 bytes to hold bit flags to signify which message has been updated and needs to be sent. Zero the bit flags in the transmit loop.

Every time through the measure loop, just check the measured value against what's in the data buffer, and if it's different update the data buffer and set it's corresponding transmit flag to true. when you break out of the measure loop, you check which messages need to be sent by looking at the transmit flags, and do your transmitting.

The caveat here is that you need to have separate buffers for single/double/multi byte midi messages, but it's pretty much the same idea. I am excluding sysex from this solution as well...

pythonic pseudo-code:
Code: 
allocate data_buffer
allocate transmit_flags
zero out data_buffer

transmit_loop:
  zero out the transmit_flags buffer

  measure_loop:
    if elapsed >= transmit_bound:
      elapsed=0
      break

    measured_values = measure() // spits out an array of key/value pairs
    
    for message_byte, data_byte in measured_values:
      index = message_byte & 0xEF               //gets rid of the leading 1 in the message byte
      if data_buffer[index] != data_byte:
        data_buffer[index] = data_byte
        transmit_flags[message_byte/8] |= 1<<(message_byte%8)       //set the corresponding transmit flag

  for message_number in range(0,128):
    message_byte = message_number |  (1<<7)
    transmit = (transmit_flags[message_byte/8]>>message_byte%8)&1      // gets the corresponding transmit flag
    if transmit:
      data_byte  = data_buffer[message_number]
      transmit_midi_message(message_byte, data_byte)


I'm sure there are probably better ways to do this, and not knowing exactly what your code is doing I can't really give a truly good answer, but this program flow should work. someone correct me if i'm wrong! :)
 
I'm actually writing exactly this right now. What I did was this.
1. If the pot has changed, store the value and mark as "Dirty"
2. In the main loop, check to see if the output buffer is clear.
3. If the output buffer is clear, send pot CCs marked as dirty and change their flag to clean.

This way, the latest value is always sent, but the buffer is not overloaded.

I'm having a different problem now tho... receiving commands with "running status" is occasionally losing the next real command that follows, resulting in stuck notes on keyboards with aftertouch. I think they fixed this in the latest version of the MIDI library, but I don't think it has been ported to Teensy yet. (2)
Edit: I tried the latest library and it worked without issue and fixed the problem.
 
Last edited:
Thank you everyone for info & participation in this. I've by no means lost interest. Had to drive 8 hours to an unexpected family funeral last week, came home w/ a 102 fever. Been down for the count. I'll try to reduce my code/hw to the minimum to demonstrate a very peculiar phenomenon that prompted the original question. I'll post back w/ code/schematic when I have something repeatable. Figured I'd start w/ a generic question & hope to catch it in the coarse net... I SHOULD know better, lol.
 
Thank you everyone for the replies & great ideas. I've been out of town for a funeral, and then sick for a week and totally out of it. I'm going to try some of the stuff here & see if it improves my current project. I'll post back.
 
Status
Not open for further replies.
Back
Top