Sending 32kb SYSEX data via usbMIDI

M1Addict

Active member
Hello!

I want to build a small tool to manage my Korg M1 with
a Teensy 4.0.

So far i got midi CC to SYSEX message translation working
for configuring the Presets.

Since i do have an audioshield with sdcard i am thinking
about how to add the feature of loading presets into the
synth via SYSEX.

I thought about simply loading the content of a SYSEX file
into an array and then simply pushing the data out via
usbMIDI.sendSysEx(32000, sysexfiledata);

Before i dive into this i thought it would be wise to ask here
if that would work or if there is another solution to this.


below is working code for controlling korg M1 preset parameters.
not really related to the question i guess.

Code:
#include "korg-M1_sysex.h" // arrays with sysex data
static uint8_t SYSEX_START = 0xF0; // start of system exclusive message 
static uint8_t SYSEX_END = 0xF7;   // end of system exclusive message
static uint8_t KORG_ID = 0x42;     // Korg
static uint8_t KORG_CHANNEL = 0x30;
static uint8_t KORG_MODEL = 0x19;  // Korg M1
static uint8_t KORG_PARAM_CHANGE = 0x41;
static uint8_t NEGATIVE_VALUE = 0x7F;
static uint8_t POSITIVE_VALUE = 0x00;
uint8_t positive_or_negative = POSITIVE_VALUE;
uint8_t parameter_page = 0;
uint8_t parameter_position=0;
char* parameter_name;
byte last_cc_message[128];
bool CHAOS_MODE=0;
                    
void setup() {
  usbMIDI.setHandleControlChange(HandleControllMessage); // set handle control messages
  usbMIDI.setHandleProgramChange(HandleProgramChange);
  usbMIDI.setHandleNoteOn(HandleNoteOnMessage);
  usbMIDI.setHandleNoteOff(HandleNoteOffMessage);
  Serial.begin(115200); while (!Serial) {;} // wait for serial
}

void loop() { usbMIDI.read(); } // read the USB MIDI bus every loop

void HandleControllMessage(byte channel, byte controller, byte value) {
// check for CHAOS_MODE Setting
if (channel == 6 && controller == 66){
  switch (value) {
    case 127:
           CHAOS_MODE  = 1;
           Serial.println("CHAOS MODE ACTIVATED !!!!");
    break;
    default:
           CHAOS_MODE  = 0;
           Serial.println("CHAOS MODE OFF");
}
}

 
// filter out cc 120 - 127 and forward it
if (controller >= 120 || controller == 64){
  usbMIDI.sendControlChange(0x01, controller, value);
  }
else if (last_cc_message[controller] == value && !CHAOS_MODE){
    Serial.print(channel); Serial.print(" ");
    Serial.print(controller); Serial.print(" ");
    Serial.print(value); Serial.print("  ");
    Serial.println("DUPLICATE MESSAGE");
}
else { 
    last_cc_message[controller] = value;
    Serial.print(channel); Serial.print(" ");
    Serial.print(controller); Serial.print(" ");
    Serial.print(value); Serial.print(" = ");
    controller = (controller - 1); // cc1 == array[0], cc128 == array[127]

switch (channel) {
    case 2: // single osclillator
            parameter_name =  sysex_single_name[controller];
            parameter_page = sysex_single_parameter_page[controller];
            parameter_position = sysex_single_parameter_position[controller];
            //   -/+ controller negativ value
            if (sysex_single_val_start[controller] == NEGATIVE_VALUE && value < 63)   
            {
              value = map(value, 0, 62, sysex_single_val_end[controller], 0);
              value = 128 - value;
              positive_or_negative = NEGATIVE_VALUE;
            }
            // -/+ controller positiv value
              else if (sysex_single_val_start[controller] == NEGATIVE_VALUE) {  
              value = map(value, 63, 127, 0, sysex_single_val_end[controller]);
              positive_or_negative = POSITIVE_VALUE;
            }
            // full range 0-x controller
              else if (sysex_single_val_start[controller] == POSITIVE_VALUE) {   
              value =   map(value, 0, 127, 0, sysex_single_val_end[controller]);
              positive_or_negative = POSITIVE_VALUE;
            }
    break;
    case 3: // double osclillator part 1
            parameter_name =  sysex_double1_name[controller];
            parameter_page = sysex_double1_parameter_page[controller];
            parameter_position = sysex_double1_parameter_position[controller];
            // -/+ controller negativ value
            if (sysex_double1_val_start[controller] == NEGATIVE_VALUE && value < 63)   
            {
              value = map(value, 0, 62, sysex_double1_val_end[controller], 0);
              value = 128 - value;
              positive_or_negative = NEGATIVE_VALUE;
            }
            // -/+ controller positiv value
              else if (sysex_double1_val_start[controller] == NEGATIVE_VALUE) {  
              value = map(value, 63, 127, 0, sysex_double1_val_end[controller]);
              positive_or_negative = POSITIVE_VALUE;
            }
            // full range 0-x controller
              else if (sysex_double1_val_start[controller] == POSITIVE_VALUE) {   
              value =   map(value, 0, 127, 0, sysex_double1_val_end[controller]);
              positive_or_negative = POSITIVE_VALUE;
            }
      break;
      case 4: // double osclillator part 2
            parameter_name =  sysex_double2_name[controller];
            parameter_page = sysex_double2_parameter_page[controller];
            parameter_position = sysex_double2_parameter_position[controller];
            // -/+ controller negativ value
            if (sysex_double2_val_start[controller] == NEGATIVE_VALUE && value < 63)   
            {
              value = map(value, 0, 62, sysex_double2_val_end[controller], 0);
              value = 128 - value;
              positive_or_negative = NEGATIVE_VALUE;
            }
            // -/+ controller positiv value
              else if (sysex_double2_val_start[controller] == NEGATIVE_VALUE) {  
              value = map(value, 63, 127, 0, sysex_double2_val_end[controller]);
              positive_or_negative = POSITIVE_VALUE;
            }
            // full range 0-x controller
              else if (sysex_double2_val_start[controller] == POSITIVE_VALUE) {   
              value =   map(value, 0, 127, 0, sysex_double2_val_end[controller]);
              positive_or_negative = POSITIVE_VALUE;
            }
      break;
  }

uint8_t sysex[] = {SYSEX_START, \
                    KORG_ID, \
                    KORG_CHANNEL, \
                    KORG_MODEL, \
                    KORG_PARAM_CHANGE,\
                    parameter_page, \
                    parameter_position,\
                    value, \
                    positive_or_negative, \
                    SYSEX_END};

for (int element : sysex){Serial.print(element < 16 ? "0" : "");Serial.print(element, HEX);Serial.print(" ");}
//Serial.println();
Serial.print(" ::: ");
Serial.println(parameter_name);
usbMIDI.sendSysEx(10, sysex);

//delay(1); // 1ms pause to not stress out the receiver, just a random value, lower may work - untested delayMicroseconds(100)
  
 }
}

void HandleProgramChange(byte channel, byte program) { usbMIDI.sendProgramChange(program, 0x01); }
void HandleNoteOnMessage(byte channel, byte pitch, byte velocity) { usbMIDI.sendNoteOn(pitch, velocity, 0x01); }
void HandleNoteOffMessage(byte channel, byte pitch, byte velocity) { usbMIDI.sendNoteOff(pitch, velocity, 0x01);}
 

Attachments

  • korg-M1_sysex.h
    8.7 KB · Views: 32
  • cc2sysex_4_channel_midi1.ino
    7.1 KB · Views: 25
Last edited:
I tried putting a test array containing a SYSEX dump into my header file.
for trying out usbMIDI.sendSysEx(31520, sysexfile_data)


Code:
char sysexfile_data[31520] = {\
0x42,0xf0,0x19,0x30,0x00,0x4d,0x4d,0x00,0x73,0x69,\
0x65,0x74,0x42,0x72,0x75,0x00,0x65,0x74,0x17,0x01,\
0x32,0x00,0x32,0x00,0x28,0x28,0x00,0x00,0x63,0x1f,\

---snip---

0x56,0x1a,0x00,0x05,0x00,0x00,0x00,0x00,0x58,0x1b,\
0x00,0x05,0x00,0x00,0x00,0x00,0x5a,0x1c,0x00,0x05,\
0x00,0x00,0x00,0x00,0x5c,0x1d,0x00,0x05,0x00,0x00,\
0x00,0x00,0x5e,0x1e,0x00,0x05,0x00,0x00,0xf7};

unfortunatly i guess size is an issue here.

just by adding the BIG array to the headerfile the sketch does not
compile any more.

The error seems to be semi related to the issue:

Code:
cc2sysex_4_channel_midi1: In function 'void setup()':
cc2sysex_4_channel_midi1:44: error: 'HandleControllMessage' was not declared in this scope
   usbMIDI.setHandleControlChange(HandleControllMessage); // set handle control messages
                                  ^
cc2sysex_4_channel_midi1:45: error: 'HandleProgramChange' was not declared in this scope
   usbMIDI.setHandleProgramChange(HandleProgramChange);
                                  ^
cc2sysex_4_channel_midi1:46: error: 'HandleNoteOnMessage' was not declared in this scope
   usbMIDI.setHandleNoteOn(HandleNoteOnMessage);
                           ^
cc2sysex_4_channel_midi1:47: error: 'HandleNoteOffMessage' was not declared in this scope
   usbMIDI.setHandleNoteOff(HandleNoteOffMessage);
                            ^
'HandleControllMessage' was not declared in this scope

removing the big array makes the code compile.

I assume there is a array size limit or something...

anyone any ideas?

thanks
 
I tried putting a test array containing a SYSEX dump into my header file.
for trying out usbMIDI.sendSysEx(31520, sysexfile_data)

Code:
char sysexfile_data[31520] = {\
0x42,0xf0,0x19,0x30,0x00,0x4d,0x4d,0x00,0x73,0x69,\
0x65,0x74,0x42,0x72,0x75,0x00,0x65,0x74,0x17,0x01,\
0x32,0x00,0x32,0x00,0x28,0x28,0x00,0x00,0x63,0x1f,\

removing the big array makes the code compile. I assume there is a array size limit or something. anyone any ideas?

It's less than 32K bytes, so space is probably not an issue, but you need to remove the '\' characters at the end of each line. That's a syntax error. You only need that when you are extending a literal string, or a macro. Should look like this:

Code:
char sysexfile_data[31520] = {
0x42,0xf0,0x19,0x30,0x00,0x4d,0x4d,0x00,0x73,0x69,
0x65,0x74,0x42,0x72,0x75,0x00,0x65,0x74,0x17,0x01,
0x32,0x00,0x32,0x00,0x28,0x28,0x00,0x00,0x63,0x1f,
...
};
 
btw. this is how to make a sysex file into easy to fix array data(remove the last comma):
Code:
hexdump -v $SYSEX_FILE  | awk '{print $2$3$4$5$6$7$8$9}' | fold -w 2 | sed 's/^/0x/g' | sed 's/$/\,/g'| tr -d '\n' | fold -w 50
 
SORRY, The line above is rubbish and does not produce data suitable for usage in an array!

The bytes look nice but are all mixed up :) F0 42 becomes 42 F0 and so on :)

Havent noticed the mixed up bytes since i havent tried the sysex transfer yet.


This one is right:

Code:
hexdump -ve '1/1 " 0x%.2x,"' $SYSEX_FILE  | fold -w 60

manual removal of the last comma is still necessary.
 
manual removal of the last comma is still necessary.

If you mean the last comma in the list of initializers, as shown below, it's okay to leave it in, for both C and C++.

Code:
char example_data[30] = {
0x42,0xf0,0x19,0x30,0x00,0x4d,0x4d,0x00,0x73,0x69,
0x65,0x74,0x42,0x72,0x75,0x00,0x65,0x74,0x17,0x01,
0x32,0x00,0x32,0x00,0x28,0x28,0x00,0x00,0x63,0x1f,
};
 
Thanks!

Smarter every day :)

Also. regarding the "\" problem :)

They are legal for all the other small arrays i used here :)

Only the Big Arrays seem to somehow confuse the compile process
when putting an escape character before the newline.

EDIT: now that i am thinking about it that makes total sense.
if the newline escapes are removed during translation/compilation
there - out of nothing - is a huge line there.
i can think of many things that cant handle lines that long.
 
Last edited:
hello again! *usbMIDI.sendSysEx Rate Limiting, any ideas?

I got time to try sending the fixed sysex data via usbMIDI.sendSysEx(31520, sysexfile_data).

After the first 256 bytes are received - errors apear and the received data is much to little.

EDIT3: since 256 is such a specific number. my guess is the alsa midi buffer gets filled up.
and does not ever again catch up with the teensy.

I tried debugging this but have just found ways to silently mess up linux alsa midi
in a way that i only solved by dis- and reconnecting the teensy.
arecordmidi and aseqdump could not get anymore data after one call
to usbMIDI.sendSysEx(31520, sysexfile_data).

So i am thinking about rate limiting or chopping up the data.

Open for any suggestions :)


LAST_EDIT: I managed to completly reset the M1 preset memory by using this
work_in_progress "tool". basically what you get when the battery dies.
only one piano preset left :)

So have a SYSEX_preset file ready and be prepared for reloading the preset.

EDIT: if someone likes to try it. the channel to send CC messages to is 1 _not_ 2 for now.
i put it to 2 later, but 1 is easier for testing since i have an old midi controller and
dont want to reconfigure the midi channel for 100 CC configurations manually.

EDIT2: also for testing it with Linux you have to connect some midiports first.
for me its:
Code:
aconnect controller:id teensy:id ; aconnect teensy:id usb_midiinterface_to_korg_M1:id
 

Attachments

  • korg-M1_sysex.h
    9.2 KB · Views: 21
  • valhala_sysex.h
    187.8 KB · Views: 29
  • cc2sysex_0602.ino
    8 KB · Views: 29
Last edited:
Thanks!

Smarter every day :)

Also. regarding the "\" problem :)

They are legal for all the other small arrays i used here :)

Only the Big Arrays seem to somehow confuse the compile process
when putting an escape character before the newline.

EDIT: now that i am thinking about it that makes total sense.
if the newline escapes are removed during translation/compilation
there - out of nothing - is a huge line there.
i can think of many things that cant handle lines that long.

I don't think it's the total length of the line that matters, but rather what follows the escape character '\'.
 
Since this was easy to try out.

i removed all newlines .

Its the length of the line. that somehow produces this error.

same error without the escapes \.
see: attachment
 

Attachments

  • valhala_sysex.h
    184.7 KB · Views: 28
Since this was easy to try out.

i removed all newlines .

Its the length of the line. that somehow produces this error.

same error without the escapes \.
see: attachment

I get no error on that using Arduino IDE, either by #include or copying the long line into my sketch.
 
Maybe we got different compiler/toolchain versions?

Arduino: 1.8.13
Linux: Ubuntu 22.04.1 LTS
arm-none-eabi-g++ (GNU Tools for ARM Embedded Processors) 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496]

Did you copy the array into any program or did you use cc2sysex_0602.ino ?

Code:
/home/me/arduino-1.8.13/arduino-builder -dump-prefs -logger=machine -hardware /home/me/arduino-1.8.13/hardware -tools /home/me/arduino-1.8.13/tools-builder -tools /home/me/arduino-1.8.13/hardware/tools/avr -built-in-libraries /home/me/arduino-1.8.13/libraries -libraries /home/me/Arduino/libraries -fqbn=teensy:avr:teensy40:usb=serialmidi,speed=600,opt=o2std,keys=en-us -vid-pid=16C0_0489 -ide-version=10813 -build-path /tmp/arduino_build_614388 -warnings=all -build-cache /tmp/arduino_cache_178930 -prefs=runtime.tools.teensy-tools.path=/home/me/.arduino15/packages/teensy/tools/teensy-tools/1.57.1 -prefs=runtime.tools.teensy-tools-1.57.1.path=/home/me/.arduino15/packages/teensy/tools/teensy-tools/1.57.1 -prefs=runtime.tools.teensy-compile.path=/home/me/.arduino15/packages/teensy/tools/teensy-compile/5.4.1 -prefs=runtime.tools.teensy-compile-5.4.1.path=/home/me/.arduino15/packages/teensy/tools/teensy-compile/5.4.1 -verbose /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino
/home/me/arduino-1.8.13/arduino-builder -compile -logger=machine -hardware /home/me/arduino-1.8.13/hardware -tools /home/me/arduino-1.8.13/tools-builder -tools /home/me/arduino-1.8.13/hardware/tools/avr -built-in-libraries /home/me/arduino-1.8.13/libraries -libraries /home/me/Arduino/libraries -fqbn=teensy:avr:teensy40:usb=serialmidi,speed=600,opt=o2std,keys=en-us -vid-pid=16C0_0489 -ide-version=10813 -build-path /tmp/arduino_build_614388 -warnings=all -build-cache /tmp/arduino_cache_178930 -prefs=runtime.tools.teensy-tools.path=/home/me/.arduino15/packages/teensy/tools/teensy-tools/1.57.1 -prefs=runtime.tools.teensy-tools-1.57.1.path=/home/me/.arduino15/packages/teensy/tools/teensy-tools/1.57.1 -prefs=runtime.tools.teensy-compile.path=/home/me/.arduino15/packages/teensy/tools/teensy-compile/5.4.1 -prefs=runtime.tools.teensy-compile-5.4.1.path=/home/me/.arduino15/packages/teensy/tools/teensy-compile/5.4.1 -verbose /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino
Using board 'teensy40' from platform in folder: /home/me/arduino-1.8.13/hardware/teensy/avr
Using core 'teensy4' from platform in folder: /home/me/arduino-1.8.13/hardware/teensy/avr
Detecting libraries used...
/home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /dev/null -DARDUINO_LIB_DISCOVERY_PHASE
Generating function prototypes...
/home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /tmp/arduino_build_614388/preproc/ctags_target_for_gcc_minus_e.cpp -DARDUINO_LIB_DISCOVERY_PHASE
/home/me/arduino-1.8.13/tools-builder/ctags/5.8-arduino11/ctags -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives /tmp/arduino_build_614388/preproc/ctags_target_for_gcc_minus_e.cpp
Compiling sketch...
/home/me/arduino-1.8.13/hardware/teensy/../tools/precompile_helper /home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388 /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -x c++-header -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/pch/Arduino.h -o /tmp/arduino_build_614388/pch/Arduino.h.gch
Using previously compiled file: /tmp/arduino_build_614388/pch/Arduino.h.gch
/home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/tmp/arduino_build_614388/pch -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o
cc2sysex_longline: In function 'void setup()':
cc2sysex_longline:47: error: 'HandleControllMessage' was not declared in this scope
   usbMIDI.setHandleControlChange(HandleControllMessage); // set handle control messages
                                  ^
cc2sysex_longline:48: error: 'HandleProgramChange' was not declared in this scope
   usbMIDI.setHandleProgramChange(HandleProgramChange);
                                  ^
cc2sysex_longline:49: error: 'HandleNoteOnMessage' was not declared in this scope
   usbMIDI.setHandleNoteOn(HandleNoteOnMessage);
                           ^
cc2sysex_longline:50: error: 'HandleNoteOffMessage' was not declared in this scope
   usbMIDI.setHandleNoteOff(HandleNoteOffMessage);
                            ^
'HandleControllMessage' was not declared in this scope
 
i tried to compile with some extra verbosity....
nothing to see here.

Code:
/home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -v -save-temps -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/tmp/arduino_build_614388/pch -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o
Using built-in specs.
COLLECT_GCC=/home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++
Target: arm-none-eabi
Configured with: /home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/src/gcc/configure --target=arm-none-eabi --prefix=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native --libexecdir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/lib --infodir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/share/doc/gcc-arm-none-eabi/info --mandir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/share/doc/gcc-arm-none-eabi/man --htmldir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/share/doc/gcc-arm-none-eabi/html --pdfdir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-newlib --with-headers=yes --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/arm-none-eabi --build=x86_64-linux-gnu --host=x86_64-linux-gnu --with-gmp=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-mpfr=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-mpc=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-isl=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-cloog=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-libelf=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='GNU Tools for ARM Embedded Processors' --with-multilib-list=armv6-m,armv7-m,armv7e-m,armv7-r,armv8-m.base,armv8-m.main
Thread model: single
gcc version 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496] (GNU Tools for ARM Embedded Processors) 
COLLECT_GCC_OPTIONS='-v' '-save-temps' '-c' '-O2' '-g' '-Wall' '-ffunction-sections' '-fdata-sections' '-nostdlib' '-MMD' '-std=gnu++14' '-fno-exceptions' '-fpermissive' '-fno-rtti' '-fno-threadsafe-statics' '-felide-constructors' '-Wno-error=narrowing' '-mthumb' '-mcpu=cortex-m7' '-mfloat-abi=hard' '-mfpu=fpv5-d16' '-D' '__IMXRT1062__' '-D' 'TEENSYDUINO=157' '-D' 'ARDUINO=10813' '-D' 'ARDUINO_TEENSY40' '-D' 'F_CPU=600000000' '-D' 'USB_MIDI_SERIAL' '-D' 'LAYOUT_US_ENGLISH' '-I' '/tmp/arduino_build_614388/pch' '-I' '/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4' '-o' '/tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o'
 /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/cc1plus -E -quiet -v -I /tmp/arduino_build_614388/pch -I /home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 -imultilib armv7e-m/fpu/fpv5-d16 -iprefix /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/ -isysroot /home/me/arduino-1.8.13/hardware/tools/arm/bin/../arm-none-eabi -MMD /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.d -MQ /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o -D__USES_INITFINI__ -D __IMXRT1062__ -D TEENSYDUINO=157 -D ARDUINO=10813 -D ARDUINO_TEENSY40 -D F_CPU=600000000 -D USB_MIDI_SERIAL -D LAYOUT_US_ENGLISH /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -std=gnu++14 -Wall -Wno-error=narrowing -ffunction-sections -fdata-sections -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -g -fworking-directory -O2 -fpch-preprocess -o cc2sysex_longline.ino.ii
ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1"
ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1/arm-none-eabi/armv7e-m/fpu/fpv5-d16"
ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1/backward"
ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/include"
ignoring nonexistent directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../arm-none-eabi/usr/local/include"
ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/include-fixed"
ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include"
ignoring nonexistent directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../arm-none-eabi/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /tmp/arduino_build_614388/pch
 /home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4
 /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1
 /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1/arm-none-eabi/armv7e-m/fpu/fpv5-d16
 /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1/backward
 /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/include
 /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/include-fixed
 /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include
End of search list.
COLLECT_GCC_OPTIONS='-v' '-save-temps' '-c' '-O2' '-g' '-Wall' '-ffunction-sections' '-fdata-sections' '-nostdlib' '-MMD' '-std=gnu++14' '-fno-exceptions' '-fpermissive' '-fno-rtti' '-fno-threadsafe-statics' '-felide-constructors' '-Wno-error=narrowing' '-mthumb' '-mcpu=cortex-m7' '-mfloat-abi=hard' '-mfpu=fpv5-d16' '-D' '__IMXRT1062__' '-D' 'TEENSYDUINO=157' '-D' 'ARDUINO=10813' '-D' 'ARDUINO_TEENSY40' '-D' 'F_CPU=600000000' '-D' 'USB_MIDI_SERIAL' '-D' 'LAYOUT_US_ENGLISH' '-I' '/tmp/arduino_build_614388/pch' '-I' '/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4' '-o' '/tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o'
 /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/cc1plus -fpreprocessed cc2sysex_longline.ino.ii -quiet -dumpbase cc2sysex_longline.ino.cpp -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -auxbase-strip /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o -g -O2 -Wall -Wno-error=narrowing -std=gnu++14 -version -ffunction-sections -fdata-sections -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -o cc2sysex_longline.ino.s
GNU C++14 (GNU Tools for ARM Embedded Processors) version 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496] (arm-none-eabi)
	compiled by GNU C version 4.3.2, GMP version 4.3.2, MPFR version 2.4.2, MPC version 0.8.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU C++14 (GNU Tools for ARM Embedded Processors) version 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496] (arm-none-eabi)
	compiled by GNU C version 4.3.2, GMP version 4.3.2, MPFR version 2.4.2, MPC version 0.8.1
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: f056b3585372add73ff3e1df6cbf929e
/home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino: In function 'void setup()':
/home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino:47:34: error: 'HandleControllMessage' was not declared in this scope
   usbMIDI.setHandleControlChange(HandleControllMessage); // set handle control messages
                                  ^
/home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino:48:34: error: 'HandleProgramChange' was not declared in this scope
   usbMIDI.setHandleProgramChange(HandleProgramChange);
                                  ^
/home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino:49:27: error: 'HandleNoteOnMessage' was not declared in this scope
   usbMIDI.setHandleNoteOn(HandleNoteOnMessage);
                           ^
/home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino:50:28: error: 'HandleNoteOffMessage' was not declared in this scope
   usbMIDI.setHandleNoteOff(HandleNoteOffMessage);

EDIT: Did an strace on both compiler runs. somethings going on. but its hard to work with those long lines produced by:
Code:
strace -v -s 1000000 -f /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -v -save-temps -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/tmp/arduino_build_614388/pch -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o &> out_broken.txt

and

Code:
strace -v -s 1000000 /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -v -save-temps -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/tmp/arduino_build_728750/pch -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_728750/sketch/cc2sysex_0602.ino.cpp -o /tmp/arduino_build_728750/sketch/cc2sysex_0602.ino.cpp.o &> out_working.txt

gonna give up on this for now. cant even upload a tar.gz with my to traces :)
 
Last edited:
Doing something similar with a Teensy based control surface/ editor for the Roland JV-1080 / 2080. Currently pulling ~8Kb of sysex and pushing it back.

Had all sorts of synth data errors, sounds that made no sense and crashed it totally requiring factory reset. Reason was sysex was being sent too fast.

Found it necessary to monitor (Windoze user here so used MidiOx) what the synth sends after pressing that magic "send everything" button and examining the time between received blocks of sysex then re-factor my code to more or less replicate what, when and how the synth sends it.
 
Oh, nice. How did you slow down the transfer from the teensy to the synth?

EDIT: was thinking about sending the SYSEX_DATA in 256 byte chunks with
some bunch of microseconds delay after every block.
 
Maybe we got different compiler/toolchain versions?

I'm using TD 1.58b3, with GCC 11.something, so are using different compilers, but I doubt that's the issue. Compilers don't read files by line. Bottom line, don't use an escape '\' where you don't need one.
 
jepp! ill remember that!

still i dont know what is going on.

but since im in no way qualified to continue to debug that
and meld (my tool of choice for diffs) does not like those
long strace lines im going to stop here.

thanks for looking into it.
 
Um, Took a month or so.. a bunch of bools attached to the compilation and sending of a given block. bools are tested in a timed sequence in loop(), marked false, sent and a timer set to determine when next bool is tested. So is non-blocking.
 
Some small number games...

MIDI over the default serial is 3125 bytes / second.

256bytes of sysex data fits 12 times into this one second capacity of MIDI serial. (or make it 10 times for safety)

So by measuring the time it takes to spit out 256bytes.

then multiplying that time by 10.

I got the total send time.

now subtracting that time from 1 second

should give a number that devided by 10 equals the wait time after each block of 256bytes.
 
MatrixRat:

I see. Blocking is no issue for me since i cant
control the synth whilst uploading presets anyway.

i dont care what the Synth may say since im not listening ;)
 
The M1 may only have 128 byte RX buffer and need time to digest incoming.

Just checked, for the most part I have 50 Ms between blocks. In one case, have 100 Ms or FX are screwed.
 
just measured how long a sysex preset transfer takes.
by running:

Code:
 time amidi -p korg_M1:id -s sysex_preset_31520.syx

12 seconds and 38 milli seconds it is. for the 31520byte file.

now back to the calculation.

31520 bytes / 12038 ms = 2.6 bytes/ms

2.6 bytes * 1000 = 2600 bytes / second
2600 / 256 = 10.15

id say the Korg M1 runs full midi speed :)

EDIT: which equals to sending out 256 bytes every ~100 ms
or 128 bytes every 50ms.
 
Last edited:
Back
Top