Forum Rule: Always post complete source code & details to reproduce any issue!
Page 1 of 2 1 2 LastLast
Results 1 to 25 of 34

Thread: Sending 32kb SYSEX data via usbMIDI

  1. #1

    Sending 32kb SYSEX data via usbMIDI

    Hello!

    I want to build a small tool to manage my Korg M1 with
    a Teensy 4.0.

    So far i got midi CC to SYSEX message translation working
    for configuring the Presets.

    Since i do have an audioshield with sdcard i am thinking
    about how to add the feature of loading presets into the
    synth via SYSEX.

    I thought about simply loading the content of a SYSEX file
    into an array and then simply pushing the data out via
    usbMIDI.sendSysEx(32000, sysexfiledata);

    Before i dive into this i thought it would be wise to ask here
    if that would work or if there is another solution to this.


    below is working code for controlling korg M1 preset parameters.
    not really related to the question i guess.

    Code:
    #include "korg-M1_sysex.h" // arrays with sysex data
    static uint8_t SYSEX_START = 0xF0; // start of system exclusive message 
    static uint8_t SYSEX_END = 0xF7;   // end of system exclusive message
    static uint8_t KORG_ID = 0x42;     // Korg
    static uint8_t KORG_CHANNEL = 0x30;
    static uint8_t KORG_MODEL = 0x19;  // Korg M1
    static uint8_t KORG_PARAM_CHANGE = 0x41;
    static uint8_t NEGATIVE_VALUE = 0x7F;
    static uint8_t POSITIVE_VALUE = 0x00;
    uint8_t positive_or_negative = POSITIVE_VALUE;
    uint8_t parameter_page = 0;
    uint8_t parameter_position=0;
    char* parameter_name;
    byte last_cc_message[128];
    bool CHAOS_MODE=0;
                        
    void setup() {
      usbMIDI.setHandleControlChange(HandleControllMessage); // set handle control messages
      usbMIDI.setHandleProgramChange(HandleProgramChange);
      usbMIDI.setHandleNoteOn(HandleNoteOnMessage);
      usbMIDI.setHandleNoteOff(HandleNoteOffMessage);
      Serial.begin(115200); while (!Serial) {;} // wait for serial
    }
    
    void loop() { usbMIDI.read(); } // read the USB MIDI bus every loop
    
    void HandleControllMessage(byte channel, byte controller, byte value) {
    // check for CHAOS_MODE Setting
    if (channel == 6 && controller == 66){
      switch (value) {
        case 127:
               CHAOS_MODE  = 1;
               Serial.println("CHAOS MODE ACTIVATED !!!!");
        break;
        default:
               CHAOS_MODE  = 0;
               Serial.println("CHAOS MODE OFF");
    }
    }
    
     
    // filter out cc 120 - 127 and forward it
    if (controller >= 120 || controller == 64){
      usbMIDI.sendControlChange(0x01, controller, value);
      }
    else if (last_cc_message[controller] == value && !CHAOS_MODE){
        Serial.print(channel); Serial.print(" ");
        Serial.print(controller); Serial.print(" ");
        Serial.print(value); Serial.print("  ");
        Serial.println("DUPLICATE MESSAGE");
    }
    else { 
        last_cc_message[controller] = value;
        Serial.print(channel); Serial.print(" ");
        Serial.print(controller); Serial.print(" ");
        Serial.print(value); Serial.print(" = ");
        controller = (controller - 1); // cc1 == array[0], cc128 == array[127]
    
    switch (channel) {
        case 2: // single osclillator
                parameter_name =  sysex_single_name[controller];
                parameter_page = sysex_single_parameter_page[controller];
                parameter_position = sysex_single_parameter_position[controller];
                //   -/+ controller negativ value
                if (sysex_single_val_start[controller] == NEGATIVE_VALUE && value < 63)   
                {
                  value = map(value, 0, 62, sysex_single_val_end[controller], 0);
                  value = 128 - value;
                  positive_or_negative = NEGATIVE_VALUE;
                }
                // -/+ controller positiv value
                  else if (sysex_single_val_start[controller] == NEGATIVE_VALUE) {  
                  value = map(value, 63, 127, 0, sysex_single_val_end[controller]);
                  positive_or_negative = POSITIVE_VALUE;
                }
                // full range 0-x controller
                  else if (sysex_single_val_start[controller] == POSITIVE_VALUE) {   
                  value =   map(value, 0, 127, 0, sysex_single_val_end[controller]);
                  positive_or_negative = POSITIVE_VALUE;
                }
        break;
        case 3: // double osclillator part 1
                parameter_name =  sysex_double1_name[controller];
                parameter_page = sysex_double1_parameter_page[controller];
                parameter_position = sysex_double1_parameter_position[controller];
                // -/+ controller negativ value
                if (sysex_double1_val_start[controller] == NEGATIVE_VALUE && value < 63)   
                {
                  value = map(value, 0, 62, sysex_double1_val_end[controller], 0);
                  value = 128 - value;
                  positive_or_negative = NEGATIVE_VALUE;
                }
                // -/+ controller positiv value
                  else if (sysex_double1_val_start[controller] == NEGATIVE_VALUE) {  
                  value = map(value, 63, 127, 0, sysex_double1_val_end[controller]);
                  positive_or_negative = POSITIVE_VALUE;
                }
                // full range 0-x controller
                  else if (sysex_double1_val_start[controller] == POSITIVE_VALUE) {   
                  value =   map(value, 0, 127, 0, sysex_double1_val_end[controller]);
                  positive_or_negative = POSITIVE_VALUE;
                }
          break;
          case 4: // double osclillator part 2
                parameter_name =  sysex_double2_name[controller];
                parameter_page = sysex_double2_parameter_page[controller];
                parameter_position = sysex_double2_parameter_position[controller];
                // -/+ controller negativ value
                if (sysex_double2_val_start[controller] == NEGATIVE_VALUE && value < 63)   
                {
                  value = map(value, 0, 62, sysex_double2_val_end[controller], 0);
                  value = 128 - value;
                  positive_or_negative = NEGATIVE_VALUE;
                }
                // -/+ controller positiv value
                  else if (sysex_double2_val_start[controller] == NEGATIVE_VALUE) {  
                  value = map(value, 63, 127, 0, sysex_double2_val_end[controller]);
                  positive_or_negative = POSITIVE_VALUE;
                }
                // full range 0-x controller
                  else if (sysex_double2_val_start[controller] == POSITIVE_VALUE) {   
                  value =   map(value, 0, 127, 0, sysex_double2_val_end[controller]);
                  positive_or_negative = POSITIVE_VALUE;
                }
          break;
      }
    
    uint8_t sysex[] = {SYSEX_START, \
                        KORG_ID, \
                        KORG_CHANNEL, \
                        KORG_MODEL, \
                        KORG_PARAM_CHANGE,\
                        parameter_page, \
                        parameter_position,\
                        value, \
                        positive_or_negative, \
                        SYSEX_END};
    
    for (int element : sysex){Serial.print(element < 16 ? "0" : "");Serial.print(element, HEX);Serial.print(" ");}
    //Serial.println();
    Serial.print(" ::: ");
    Serial.println(parameter_name);
    usbMIDI.sendSysEx(10, sysex);
    
    //delay(1); // 1ms pause to not stress out the receiver, just a random value, lower may work - untested delayMicroseconds(100)
      
     }
    }
    
    void HandleProgramChange(byte channel, byte program) { usbMIDI.sendProgramChange(program, 0x01); }
    void HandleNoteOnMessage(byte channel, byte pitch, byte velocity) { usbMIDI.sendNoteOn(pitch, velocity, 0x01); }
    void HandleNoteOffMessage(byte channel, byte pitch, byte velocity) { usbMIDI.sendNoteOff(pitch, velocity, 0x01);}
    Attached Files Attached Files
    Last edited by M1Addict; 02-03-2023 at 11:05 AM.

  2. #2
    I tried putting a test array containing a SYSEX dump into my header file.
    for trying out usbMIDI.sendSysEx(31520, sysexfile_data)


    Code:
    char sysexfile_data[31520] = {\
    0x42,0xf0,0x19,0x30,0x00,0x4d,0x4d,0x00,0x73,0x69,\
    0x65,0x74,0x42,0x72,0x75,0x00,0x65,0x74,0x17,0x01,\
    0x32,0x00,0x32,0x00,0x28,0x28,0x00,0x00,0x63,0x1f,\
    
    ---snip---
    
    0x56,0x1a,0x00,0x05,0x00,0x00,0x00,0x00,0x58,0x1b,\
    0x00,0x05,0x00,0x00,0x00,0x00,0x5a,0x1c,0x00,0x05,\
    0x00,0x00,0x00,0x00,0x5c,0x1d,0x00,0x05,0x00,0x00,\
    0x00,0x00,0x5e,0x1e,0x00,0x05,0x00,0x00,0xf7};
    unfortunatly i guess size is an issue here.

    just by adding the BIG array to the headerfile the sketch does not
    compile any more.

    The error seems to be semi related to the issue:

    Code:
    cc2sysex_4_channel_midi1: In function 'void setup()':
    cc2sysex_4_channel_midi1:44: error: 'HandleControllMessage' was not declared in this scope
       usbMIDI.setHandleControlChange(HandleControllMessage); // set handle control messages
                                      ^
    cc2sysex_4_channel_midi1:45: error: 'HandleProgramChange' was not declared in this scope
       usbMIDI.setHandleProgramChange(HandleProgramChange);
                                      ^
    cc2sysex_4_channel_midi1:46: error: 'HandleNoteOnMessage' was not declared in this scope
       usbMIDI.setHandleNoteOn(HandleNoteOnMessage);
                               ^
    cc2sysex_4_channel_midi1:47: error: 'HandleNoteOffMessage' was not declared in this scope
       usbMIDI.setHandleNoteOff(HandleNoteOffMessage);
                                ^
    'HandleControllMessage' was not declared in this scope
    removing the big array makes the code compile.

    I assume there is a array size limit or something...

    anyone any ideas?

    thanks

  3. #3
    forgot to post the headerfile.
    Attached Files Attached Files

  4. #4
    Quote Originally Posted by M1Addict View Post
    I tried putting a test array containing a SYSEX dump into my header file.
    for trying out usbMIDI.sendSysEx(31520, sysexfile_data)

    Code:
    char sysexfile_data[31520] = {\
    0x42,0xf0,0x19,0x30,0x00,0x4d,0x4d,0x00,0x73,0x69,\
    0x65,0x74,0x42,0x72,0x75,0x00,0x65,0x74,0x17,0x01,\
    0x32,0x00,0x32,0x00,0x28,0x28,0x00,0x00,0x63,0x1f,\
    removing the big array makes the code compile. I assume there is a array size limit or something. anyone any ideas?
    It's less than 32K bytes, so space is probably not an issue, but you need to remove the '\' characters at the end of each line. That's a syntax error. You only need that when you are extending a literal string, or a macro. Should look like this:

    Code:
    char sysexfile_data[31520] = {
    0x42,0xf0,0x19,0x30,0x00,0x4d,0x4d,0x00,0x73,0x69,
    0x65,0x74,0x42,0x72,0x75,0x00,0x65,0x74,0x17,0x01,
    0x32,0x00,0x32,0x00,0x28,0x28,0x00,0x00,0x63,0x1f,
    ...
    };

  5. #5
    Thanks alot joepasquariello That was the problem

  6. #6
    btw. this is how to make a sysex file into easy to fix array data(remove the last comma):
    Code:
    hexdump -v $SYSEX_FILE  | awk '{print $2$3$4$5$6$7$8$9}' | fold -w 2 | sed 's/^/0x/g' | sed 's/$/\,/g'| tr -d '\n' | fold -w 50

  7. #7
    SORRY, The line above is rubbish and does not produce data suitable for usage in an array!

    The bytes look nice but are all mixed up F0 42 becomes 42 F0 and so on

    Havent noticed the mixed up bytes since i havent tried the sysex transfer yet.


    This one is right:

    Code:
    hexdump -ve '1/1 " 0x%.2x,"' $SYSEX_FILE  | fold -w 60
    manual removal of the last comma is still necessary.

  8. #8
    Quote Originally Posted by M1Addict View Post
    manual removal of the last comma is still necessary.
    If you mean the last comma in the list of initializers, as shown below, it's okay to leave it in, for both C and C++.

    Code:
    char example_data[30] = {
    0x42,0xf0,0x19,0x30,0x00,0x4d,0x4d,0x00,0x73,0x69,
    0x65,0x74,0x42,0x72,0x75,0x00,0x65,0x74,0x17,0x01,
    0x32,0x00,0x32,0x00,0x28,0x28,0x00,0x00,0x63,0x1f,
    };

  9. #9
    Thanks!

    Smarter every day

    Also. regarding the "\" problem

    They are legal for all the other small arrays i used here

    Only the Big Arrays seem to somehow confuse the compile process
    when putting an escape character before the newline.

    EDIT: now that i am thinking about it that makes total sense.
    if the newline escapes are removed during translation/compilation
    there - out of nothing - is a huge line there.
    i can think of many things that cant handle lines that long.
    Last edited by M1Addict; 02-04-2023 at 10:59 PM.

  10. #10
    hello again! *usbMIDI.sendSysEx Rate Limiting, any ideas?

    I got time to try sending the fixed sysex data via usbMIDI.sendSysEx(31520, sysexfile_data).

    After the first 256 bytes are received - errors apear and the received data is much to little.

    EDIT3: since 256 is such a specific number. my guess is the alsa midi buffer gets filled up.
    and does not ever again catch up with the teensy.

    I tried debugging this but have just found ways to silently mess up linux alsa midi
    in a way that i only solved by dis- and reconnecting the teensy.
    arecordmidi and aseqdump could not get anymore data after one call
    to usbMIDI.sendSysEx(31520, sysexfile_data).

    So i am thinking about rate limiting or chopping up the data.

    Open for any suggestions :)


    LAST_EDIT: I managed to completly reset the M1 preset memory by using this
    work_in_progress "tool". basically what you get when the battery dies.
    only one piano preset left :)

    So have a SYSEX_preset file ready and be prepared for reloading the preset.

    EDIT: if someone likes to try it. the channel to send CC messages to is 1 _not_ 2 for now.
    i put it to 2 later, but 1 is easier for testing since i have an old midi controller and
    dont want to reconfigure the midi channel for 100 CC configurations manually.

    EDIT2: also for testing it with Linux you have to connect some midiports first.
    for me its:
    Code:
    aconnect controller:id teensy:id ; aconnect teensy:id usb_midiinterface_to_korg_M1:id
    Attached Files Attached Files
    Last edited by M1Addict; 02-04-2023 at 11:12 PM.

  11. #11
    Quote Originally Posted by M1Addict View Post
    Thanks!

    Smarter every day

    Also. regarding the "\" problem

    They are legal for all the other small arrays i used here

    Only the Big Arrays seem to somehow confuse the compile process
    when putting an escape character before the newline.

    EDIT: now that i am thinking about it that makes total sense.
    if the newline escapes are removed during translation/compilation
    there - out of nothing - is a huge line there.
    i can think of many things that cant handle lines that long.
    I don't think it's the total length of the line that matters, but rather what follows the escape character '\'.

  12. #12
    Since this was easy to try out.

    i removed all newlines .

    Its the length of the line. that somehow produces this error.

    same error without the escapes \.
    see: attachment
    Attached Files Attached Files

  13. #13
    Quote Originally Posted by M1Addict View Post
    Since this was easy to try out.

    i removed all newlines .

    Its the length of the line. that somehow produces this error.

    same error without the escapes \.
    see: attachment
    I get no error on that using Arduino IDE, either by #include or copying the long line into my sketch.

  14. #14
    Maybe we got different compiler/toolchain versions?

    Arduino: 1.8.13
    Linux: Ubuntu 22.04.1 LTS
    arm-none-eabi-g++ (GNU Tools for ARM Embedded Processors) 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496]

    Did you copy the array into any program or did you use cc2sysex_0602.ino ?

    Code:
    /home/me/arduino-1.8.13/arduino-builder -dump-prefs -logger=machine -hardware /home/me/arduino-1.8.13/hardware -tools /home/me/arduino-1.8.13/tools-builder -tools /home/me/arduino-1.8.13/hardware/tools/avr -built-in-libraries /home/me/arduino-1.8.13/libraries -libraries /home/me/Arduino/libraries -fqbn=teensy:avr:teensy40:usb=serialmidi,speed=600,opt=o2std,keys=en-us -vid-pid=16C0_0489 -ide-version=10813 -build-path /tmp/arduino_build_614388 -warnings=all -build-cache /tmp/arduino_cache_178930 -prefs=runtime.tools.teensy-tools.path=/home/me/.arduino15/packages/teensy/tools/teensy-tools/1.57.1 -prefs=runtime.tools.teensy-tools-1.57.1.path=/home/me/.arduino15/packages/teensy/tools/teensy-tools/1.57.1 -prefs=runtime.tools.teensy-compile.path=/home/me/.arduino15/packages/teensy/tools/teensy-compile/5.4.1 -prefs=runtime.tools.teensy-compile-5.4.1.path=/home/me/.arduino15/packages/teensy/tools/teensy-compile/5.4.1 -verbose /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino
    /home/me/arduino-1.8.13/arduino-builder -compile -logger=machine -hardware /home/me/arduino-1.8.13/hardware -tools /home/me/arduino-1.8.13/tools-builder -tools /home/me/arduino-1.8.13/hardware/tools/avr -built-in-libraries /home/me/arduino-1.8.13/libraries -libraries /home/me/Arduino/libraries -fqbn=teensy:avr:teensy40:usb=serialmidi,speed=600,opt=o2std,keys=en-us -vid-pid=16C0_0489 -ide-version=10813 -build-path /tmp/arduino_build_614388 -warnings=all -build-cache /tmp/arduino_cache_178930 -prefs=runtime.tools.teensy-tools.path=/home/me/.arduino15/packages/teensy/tools/teensy-tools/1.57.1 -prefs=runtime.tools.teensy-tools-1.57.1.path=/home/me/.arduino15/packages/teensy/tools/teensy-tools/1.57.1 -prefs=runtime.tools.teensy-compile.path=/home/me/.arduino15/packages/teensy/tools/teensy-compile/5.4.1 -prefs=runtime.tools.teensy-compile-5.4.1.path=/home/me/.arduino15/packages/teensy/tools/teensy-compile/5.4.1 -verbose /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino
    Using board 'teensy40' from platform in folder: /home/me/arduino-1.8.13/hardware/teensy/avr
    Using core 'teensy4' from platform in folder: /home/me/arduino-1.8.13/hardware/teensy/avr
    Detecting libraries used...
    /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /dev/null -DARDUINO_LIB_DISCOVERY_PHASE
    Generating function prototypes...
    /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -E -CC -x c++ -w -g -Wall -ffunction-sections -fdata-sections -nostdlib -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /tmp/arduino_build_614388/preproc/ctags_target_for_gcc_minus_e.cpp -DARDUINO_LIB_DISCOVERY_PHASE
    /home/me/arduino-1.8.13/tools-builder/ctags/5.8-arduino11/ctags -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives /tmp/arduino_build_614388/preproc/ctags_target_for_gcc_minus_e.cpp
    Compiling sketch...
    /home/me/arduino-1.8.13/hardware/teensy/../tools/precompile_helper /home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388 /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -x c++-header -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/pch/Arduino.h -o /tmp/arduino_build_614388/pch/Arduino.h.gch
    Using previously compiled file: /tmp/arduino_build_614388/pch/Arduino.h.gch
    /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/tmp/arduino_build_614388/pch -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o
    cc2sysex_longline: In function 'void setup()':
    cc2sysex_longline:47: error: 'HandleControllMessage' was not declared in this scope
       usbMIDI.setHandleControlChange(HandleControllMessage); // set handle control messages
                                      ^
    cc2sysex_longline:48: error: 'HandleProgramChange' was not declared in this scope
       usbMIDI.setHandleProgramChange(HandleProgramChange);
                                      ^
    cc2sysex_longline:49: error: 'HandleNoteOnMessage' was not declared in this scope
       usbMIDI.setHandleNoteOn(HandleNoteOnMessage);
                               ^
    cc2sysex_longline:50: error: 'HandleNoteOffMessage' was not declared in this scope
       usbMIDI.setHandleNoteOff(HandleNoteOffMessage);
                                ^
    'HandleControllMessage' was not declared in this scope

  15. #15
    i tried to compile with some extra verbosity....
    nothing to see here.

    Code:
    /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -v -save-temps -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/tmp/arduino_build_614388/pch -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o
    Using built-in specs.
    COLLECT_GCC=/home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++
    Target: arm-none-eabi
    Configured with: /home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/src/gcc/configure --target=arm-none-eabi --prefix=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native --libexecdir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/lib --infodir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/share/doc/gcc-arm-none-eabi/info --mandir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/share/doc/gcc-arm-none-eabi/man --htmldir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/share/doc/gcc-arm-none-eabi/html --pdfdir=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-newlib --with-headers=yes --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/install-native/arm-none-eabi --build=x86_64-linux-gnu --host=x86_64-linux-gnu --with-gmp=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-mpfr=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-mpc=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-isl=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-cloog=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-libelf=/home/paul/gcc-arm-none-eabi-5_4-2016q3-20160926/build-native/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='GNU Tools for ARM Embedded Processors' --with-multilib-list=armv6-m,armv7-m,armv7e-m,armv7-r,armv8-m.base,armv8-m.main
    Thread model: single
    gcc version 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496] (GNU Tools for ARM Embedded Processors) 
    COLLECT_GCC_OPTIONS='-v' '-save-temps' '-c' '-O2' '-g' '-Wall' '-ffunction-sections' '-fdata-sections' '-nostdlib' '-MMD' '-std=gnu++14' '-fno-exceptions' '-fpermissive' '-fno-rtti' '-fno-threadsafe-statics' '-felide-constructors' '-Wno-error=narrowing' '-mthumb' '-mcpu=cortex-m7' '-mfloat-abi=hard' '-mfpu=fpv5-d16' '-D' '__IMXRT1062__' '-D' 'TEENSYDUINO=157' '-D' 'ARDUINO=10813' '-D' 'ARDUINO_TEENSY40' '-D' 'F_CPU=600000000' '-D' 'USB_MIDI_SERIAL' '-D' 'LAYOUT_US_ENGLISH' '-I' '/tmp/arduino_build_614388/pch' '-I' '/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4' '-o' '/tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o'
     /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/cc1plus -E -quiet -v -I /tmp/arduino_build_614388/pch -I /home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 -imultilib armv7e-m/fpu/fpv5-d16 -iprefix /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/ -isysroot /home/me/arduino-1.8.13/hardware/tools/arm/bin/../arm-none-eabi -MMD /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.d -MQ /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o -D__USES_INITFINI__ -D __IMXRT1062__ -D TEENSYDUINO=157 -D ARDUINO=10813 -D ARDUINO_TEENSY40 -D F_CPU=600000000 -D USB_MIDI_SERIAL -D LAYOUT_US_ENGLISH /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -std=gnu++14 -Wall -Wno-error=narrowing -ffunction-sections -fdata-sections -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -g -fworking-directory -O2 -fpch-preprocess -o cc2sysex_longline.ino.ii
    ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1"
    ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1/arm-none-eabi/armv7e-m/fpu/fpv5-d16"
    ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1/backward"
    ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/include"
    ignoring nonexistent directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../arm-none-eabi/usr/local/include"
    ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/include-fixed"
    ignoring duplicate directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/../../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include"
    ignoring nonexistent directory "/home/me/arduino-1.8.13/hardware/tools/arm/bin/../arm-none-eabi/usr/include"
    #include "..." search starts here:
    #include <...> search starts here:
     /tmp/arduino_build_614388/pch
     /home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4
     /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1
     /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1/arm-none-eabi/armv7e-m/fpu/fpv5-d16
     /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include/c++/5.4.1/backward
     /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/include
     /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/include-fixed
     /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/include
    End of search list.
    COLLECT_GCC_OPTIONS='-v' '-save-temps' '-c' '-O2' '-g' '-Wall' '-ffunction-sections' '-fdata-sections' '-nostdlib' '-MMD' '-std=gnu++14' '-fno-exceptions' '-fpermissive' '-fno-rtti' '-fno-threadsafe-statics' '-felide-constructors' '-Wno-error=narrowing' '-mthumb' '-mcpu=cortex-m7' '-mfloat-abi=hard' '-mfpu=fpv5-d16' '-D' '__IMXRT1062__' '-D' 'TEENSYDUINO=157' '-D' 'ARDUINO=10813' '-D' 'ARDUINO_TEENSY40' '-D' 'F_CPU=600000000' '-D' 'USB_MIDI_SERIAL' '-D' 'LAYOUT_US_ENGLISH' '-I' '/tmp/arduino_build_614388/pch' '-I' '/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4' '-o' '/tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o'
     /home/me/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/cc1plus -fpreprocessed cc2sysex_longline.ino.ii -quiet -dumpbase cc2sysex_longline.ino.cpp -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -auxbase-strip /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o -g -O2 -Wall -Wno-error=narrowing -std=gnu++14 -version -ffunction-sections -fdata-sections -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -o cc2sysex_longline.ino.s
    GNU C++14 (GNU Tools for ARM Embedded Processors) version 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496] (arm-none-eabi)
    	compiled by GNU C version 4.3.2, GMP version 4.3.2, MPFR version 2.4.2, MPC version 0.8.1
    GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
    GNU C++14 (GNU Tools for ARM Embedded Processors) version 5.4.1 20160919 (release) [ARM/embedded-5-branch revision 240496] (arm-none-eabi)
    	compiled by GNU C version 4.3.2, GMP version 4.3.2, MPFR version 2.4.2, MPC version 0.8.1
    GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
    Compiler executable checksum: f056b3585372add73ff3e1df6cbf929e
    /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino: In function 'void setup()':
    /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino:47:34: error: 'HandleControllMessage' was not declared in this scope
       usbMIDI.setHandleControlChange(HandleControllMessage); // set handle control messages
                                      ^
    /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino:48:34: error: 'HandleProgramChange' was not declared in this scope
       usbMIDI.setHandleProgramChange(HandleProgramChange);
                                      ^
    /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino:49:27: error: 'HandleNoteOnMessage' was not declared in this scope
       usbMIDI.setHandleNoteOn(HandleNoteOnMessage);
                               ^
    /home/me/Arduino/cc2sysex_longline/cc2sysex_longline.ino:50:28: error: 'HandleNoteOffMessage' was not declared in this scope
       usbMIDI.setHandleNoteOff(HandleNoteOffMessage);
    EDIT: Did an strace on both compiler runs. somethings going on. but its hard to work with those long lines produced by:
    Code:
    strace -v -s 1000000 -f /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -v -save-temps -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/tmp/arduino_build_614388/pch -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp -o /tmp/arduino_build_614388/sketch/cc2sysex_longline.ino.cpp.o &> out_broken.txt
    and

    Code:
    strace -v -s 1000000 /home/me/arduino-1.8.13/hardware/teensy/../tools/arm/bin/arm-none-eabi-g++ -v -save-temps -c -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing -mthumb -mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-d16 -D__IMXRT1062__ -DTEENSYDUINO=157 -DARDUINO=10813 -DARDUINO_TEENSY40 -DF_CPU=600000000 -DUSB_MIDI_SERIAL -DLAYOUT_US_ENGLISH -I/tmp/arduino_build_728750/pch -I/home/me/arduino-1.8.13/hardware/teensy/avr/cores/teensy4 /tmp/arduino_build_728750/sketch/cc2sysex_0602.ino.cpp -o /tmp/arduino_build_728750/sketch/cc2sysex_0602.ino.cpp.o &> out_working.txt
    gonna give up on this for now. cant even upload a tar.gz with my to traces :)
    Last edited by M1Addict; 02-05-2023 at 01:59 AM.

  16. #16
    Senior Member
    Join Date
    Aug 2019
    Location
    Melbourne Australia
    Posts
    342
    Doing something similar with a Teensy based control surface/ editor for the Roland JV-1080 / 2080. Currently pulling ~8Kb of sysex and pushing it back.

    Had all sorts of synth data errors, sounds that made no sense and crashed it totally requiring factory reset. Reason was sysex was being sent too fast.

    Found it necessary to monitor (Windoze user here so used MidiOx) what the synth sends after pressing that magic "send everything" button and examining the time between received blocks of sysex then re-factor my code to more or less replicate what, when and how the synth sends it.

  17. #17
    Oh, nice. How did you slow down the transfer from the teensy to the synth?

    EDIT: was thinking about sending the SYSEX_DATA in 256 byte chunks with
    some bunch of microseconds delay after every block.

  18. #18
    Quote Originally Posted by M1Addict View Post
    Maybe we got different compiler/toolchain versions?
    I'm using TD 1.58b3, with GCC 11.something, so are using different compilers, but I doubt that's the issue. Compilers don't read files by line. Bottom line, don't use an escape '\' where you don't need one.

  19. #19
    jepp! ill remember that!

    still i dont know what is going on.

    but since im in no way qualified to continue to debug that
    and meld (my tool of choice for diffs) does not like those
    long strace lines im going to stop here.

    thanks for looking into it.

  20. #20
    Senior Member
    Join Date
    Aug 2019
    Location
    Melbourne Australia
    Posts
    342
    Um, Took a month or so.. a bunch of bools attached to the compilation and sending of a given block. bools are tested in a timed sequence in loop(), marked false, sent and a timer set to determine when next bool is tested. So is non-blocking.

  21. #21
    Some small number games...

    MIDI over the default serial is 3125 bytes / second.

    256bytes of sysex data fits 12 times into this one second capacity of MIDI serial. (or make it 10 times for safety)

    So by measuring the time it takes to spit out 256bytes.

    then multiplying that time by 10.

    I got the total send time.

    now subtracting that time from 1 second

    should give a number that devided by 10 equals the wait time after each block of 256bytes.

  22. #22
    MatrixRat:

    I see. Blocking is no issue for me since i cant
    control the synth whilst uploading presets anyway.

    i dont care what the Synth may say since im not listening

  23. #23
    Senior Member
    Join Date
    Aug 2019
    Location
    Melbourne Australia
    Posts
    342
    The M1 may only have 128 byte RX buffer and need time to digest incoming.

    Just checked, for the most part I have 50 Ms between blocks. In one case, have 100 Ms or FX are screwed.

  24. #24
    This may well be the next pit i dive into

  25. #25
    just measured how long a sysex preset transfer takes.
    by running:

    Code:
     time amidi -p korg_M1:id -s sysex_preset_31520.syx
    12 seconds and 38 milli seconds it is. for the 31520byte file.

    now back to the calculation.

    31520 bytes / 12038 ms = 2.6 bytes/ms

    2.6 bytes * 1000 = 2600 bytes / second
    2600 / 256 = 10.15

    id say the Korg M1 runs full midi speed

    EDIT: which equals to sending out 256 bytes every ~100 ms
    or 128 bytes every 50ms.
    Last edited by M1Addict; 02-05-2023 at 03:06 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •