memcpy leads to teensy USB disconnect - any ideas?

[M1Addict]

Hello!

Im playing around with copying data from a large u_int8 array into a small u_int8 buffer.

the large arrays size is 31520 bytes, the buffer is 256 bytes wide.

Seems that my code does something really bad.

it compiles without warnings.

but gets the teensy into a RESET LOOP.

Nothing ever gets printet to the serial.

anyone an idea where i am beeing bad to the teensy?

#include "valhala_sysex.h" // arrays with presets

void setup()
{
  Serial.begin(115200); while (!Serial) {;} // wait for serial

  // get lenghts of the sysexdata and the midi buffer
  static uint16_t SYSEX_PRESET_LENGTH = sizeof(Valhala_B102)/sizeof(Valhala_B102[0]);
  static uint8_t MIDI_BUFFER_LENGTH = sizeof(midi_buffer)/sizeof(midi_buffer[0]);
  
  uint16_t sysex_preset_offset=0;
  
  Serial.println("starting buffer test");
  
while ( sysex_preset_offset < SYSEX_PRESET_LENGTH)
  {
  memcpy(midi_buffer, Valhala_B102 + sysex_preset_offset, SYSEX_PRESET_LENGTH);
  sysex_preset_offset = sysex_preset_offset + MIDI_BUFFER_LENGTH;
  for (int element : midi_buffer)
   {
    Serial.print(element < 16 ? "0" : "");
    Serial.print(element, HEX);
    Serial.print(" ");
   }
  delay(100);
  }
}

void loop()
{
}

[kd5rxt-mark]

Hello!

Im playing around with copying data from a large u_int8 array into a small u_int8 buffer.

the large arrays size is 31520 bytes, the buffer is 256 bytes wide.

Seems that my code does something really bad.

it compiles without warnings.

but gets the teensy into a RESET LOOP.

Nothing ever gets printet to the serial.

anyone an idea where i am beeing bad to the teensy?

Should the length to be copied in your memcpy call be MIDI_BUFFER_LENGTH instead of SYSEX_PRESET_LENGTH (assuming that the MIDI buffer is smaller than the total SYSEX array) ??

Maybe try adding the following to setup() to aid in narrowing down where the crash is occurring (this capability provides info gathered during the crash, but reported as part of the next boot):

   if (CrashReport) {
      Serial.print(CrashReport);
   }

And <here's> a link with an example of how to use addr2line.

Hope that helps . . .

Mark J Culross
KD5RXT

[M1Addict]

Thanks, i tried but still no output on the serial

void setup()
{
  Serial.begin(115200); while (!Serial) {;} // wait for serial
  if (CrashReport) {
      Serial.print(CrashReport);
   }
...
...

[joepasquariello]

Im playing around with copying data from a large u_int8 array into a small u_int8 buffer. the large arrays size is 31520 bytes, the buffer is 256 bytes wide. Seems that my code does something really bad. it compiles without warnings. but gets the teensy into a RESET LOOP. Nothing ever gets printet to the serial. anyone an idea where i am beeing bad to the teensy?

In your call to memcpy(), the 1st argument is the destination (midi_buffer), the 2nd argument is the source (sysex_buffer), and the 3rd argument is the number of bytes to copy. You are passing in the size of the sysex_buffer, which is overwriting the end of the midi buffer.

[kd5rxt-mark]

Sorry, I may have been editing my post at the same time as you were replying, so you may not have seen the question about the memcpy length parameter.

Since the CrashReport is saved from the previous crash, load/run it once to get it to rebooting (crashing), then comment out the memcpy line, then load/run it again to get the report (assuming that the crash occurs as a result of the memcpy command).

Mark J Culross
KD5RXT

[M1Addict]

ok, changing the size does help in that it doesnt reset the teensy anymore.

EDIT: joepasquariello, kd5rxt-mark : So you are totally right that is wrong.
i did miss that cause it made no difference in mey small test program.

but i get only 0x00. aka, the empty buffer array.

 memcpy(midi_buffer, Valhala_B102 + sysex_preset_offset, MIDI_BUFFER_LENGTH);

the code below was my test. does what it should. but uses just unit8 and less data in the arrays.

void setup()
{
  Serial.begin(115200);
  const uint8_t  src[39] ={0xf0, 0x42, 0x30, 0x19, 0x4d, 0x00, 0x00, 0x4d, 0x69, 0x73,
                           0x78, 0x45, 0x65, 0x70, 0x72, 0x00, 0x6d, 0x69, 0x17, 0x01,
                           0x32, 0x09, 0x32, 0x00, 0x2e, 0x2e, 0x00, 0x00, 0x63, 0x1f,
                           0x43, 0x01, 0x00, 0x03, 0x00, 0x00, 0x00, 0x0a, 0x74};
                           
  uint16_t arrayLength = sizeof(src)/sizeof(src[0]);
  uint8_t dest[10]={0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,};
  uint8_t bufferLength = sizeof(dest)/sizeof(dest[0]);
  uint16_t src_offset=0;
  
  
while ( src_offset < arrayLength)
  {
  memcpy(dest, src + src_offset, arrayLength);
  src_offset = src_offset + bufferLength;
  delay(1000);
 // Serial.print("After memcpy dest = ");
  for (int element : dest)
   {
    Serial.print(element < 16 ? "0" : "");
    Serial.print(element, HEX);
    Serial.print(" ");
    Serial.print("");
   }
  Serial.println("");
  delay(100);
 }
}

void loop()
{
}

it also works when the right buffersize is used:

..
  memcpy(dest, src + src_offset, bufferLength);
..

[M1Addict]

Just noticed that my midi buffer was 10 bytes short.

i fixed that and added some 0xff on the end and on the beginning of the array as markers.

Still memcp doesnt seem to work and doesnt overwrite the buffer.

FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF 
NEXT BLOCK

EDIT: when i manually set

 static uint16_t SYSEX_PRESET_LENGTH = 31520 //+ 77;
 static uint8_t MIDI_BUFFER_LENGTH = 255;

kind of works.. but i got the last byte of the empty buffer printed
and also its 77 chars short... cant wrap my head around it.

[joepasquariello]

when i manually set

 static uint16_t SYSEX_PRESET_LENGTH = 31520 //+ 77;
 static uint8_t MIDI_BUFFER_LENGTH = 255;

kind of works.. but i got the last byte of the empty buffer printed
and also its 77 chars short... cant wrap my head around it.

Try adding these two lines after you compute the lengths of the buffers. With this information, can you see what's wrong?

  Serial.println( SYSEX_PRESET_LENGTH );
  Serial.println( MIDI_BUFFER_LENGTH );

[M1Addict]

Ok, so far. it almost works when i use a buffer size of 128.

Still having problems with the padding. and "random" values
following the last byte of the source array.

1B 58 05 00 00 00 00 00 1C 5A 05 00 00 00 00 00 1D 5C 05 00 00 00 00 00 1E 5E 05 00 00 00 00 F7 12 01 00 02 EF 02 01 40 C0 16 89 04 79 02 01 02 03 01 00 00 12 03 31 00 32 00 32 00 36 00 33 00 34 00 33 00 30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

commentet:

1B 58 05 00 00 00 00 00 1C 5A 05 00 00 00 00 00 1D 5C 05 00 00 00 00 00 1E 5E 05 00 00 00 00 F7 [HERE IS THE END OF THE SOURCE ARRAY]
12 01 00 02 EF 02 01 40 C0 16 89 04 79 02 01 02 03 01 00 00 12 03 31 00 32 00 32 00 36 00 33 00 34 00 33 00 30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

[M1Addict]

joepasquariello: Did it. Its ok. prints out the same numbers. so i revert it to non static declaration.

31520
128

the garbage after the last byte is also the same. as with fixed values for the array lengths.

[M1Addict]

ITS WORKING !!!

data array gets read into 128 byte buffers.

if the buffer size is bigger then the remaining data from the data array
the missing bytes are padded with 0x00. (overwriting the garbage that was there after memcp)

If anyone has a an idea why this does not work with a 256byte buffer id be happy to hear.

#include "valhala_sysex.h" // arrays with presets

void setup()
{
  Serial.begin(115200); while (!Serial) {;} // wait for serial
  if (CrashReport) {
      Serial.print(CrashReport);
   }
  // get lenghts of the sysexdata and the midi buffer
  static uint16_t SYSEX_PRESET_LENGTH = sizeof(Valhala_B102)/sizeof(Valhala_B102[0]);
  static uint8_t MIDI_BUFFER_LENGTH = sizeof(midi_buffer)/sizeof(midi_buffer[0]);
  uint16_t sysex_preset_offset=0;

  Serial.println( SYSEX_PRESET_LENGTH );
  Serial.println( MIDI_BUFFER_LENGTH );
  
  delay(5000);
  Serial.println("starting buffer test");

  
while ( sysex_preset_offset < SYSEX_PRESET_LENGTH)
  {
  memcpy(midi_buffer, Valhala_B102 + sysex_preset_offset, MIDI_BUFFER_LENGTH);
  sysex_preset_offset = sysex_preset_offset + MIDI_BUFFER_LENGTH;

  if ( (sysex_preset_offset - SYSEX_PRESET_LENGTH) > 0)
  { 
    int padding_bytes = sysex_preset_offset - SYSEX_PRESET_LENGTH;
    int padding_offset = MIDI_BUFFER_LENGTH - padding_bytes;
    while (padding_offset < MIDI_BUFFER_LENGTH)
      {
      midi_buffer[padding_offset]=0x00;
      padding_offset = padding_offset + 1;
      }
    }
  
  for (int element : midi_buffer)
   {
    Serial.print(element < 16 ? "0" : "");
 //   Serial.print("0x");
    Serial.print(element, HEX);
    Serial.print(", ");
   }
  
  Serial.print("");
  Serial.println(sysex_preset_offset);
//  Serial.println("NEXT BLOCK");
//  delay(100);
  }
}

void loop()
{
}

[joepasquariello]

It doesn't work with 256 because you are using a uint8_t to hold the size of midi_buffer.

[M1Addict]

It doesn't work with 256 because you are using a uint8_t to hold the size of midi_buffer.

did that. and voila!

  static uint16_t MIDI_BUFFER_LENGTH = sizeof(midi_buffer)/sizeof(midi_buffer[0]);

Now everything works as i wanted it to

Thanks for the help!

[M1Addict]

Is there a "nicer" or more correct way to solve this problem?

Happy for any links to other solutions to this.

[joepasquariello]

Is there a "nicer" or more correct way to solve this problem?

Do you mean the problem of copying data from a large buffer to a smaller one? You had the basic idea, but you had some mistakes in your logic. I suggest you make more use of print() and println() for debugging, because that immediately revealed your issue.

[M1Addict]

Do you mean the problem of copying data from a large buffer to a smaller one? You had the basic idea, but you had some mistakes in your logic. I suggest you make more use of print() and println() for debugging, because that immediately revealed your issue.

you are right with the verbose variables. ill keep that in mind.