Mysterious Crashes with String Functions

[Aerokeith]

I've been going crazy trying to debug a problem with using some of the standard string functions, particularly strstr() and strchr(). When I use these under certain conditions (see below), my program always crashes, as indicated by lack of any output on the serial monitor and the need to press the button on the Teensy 4.0 to load any subsequent programs.

Configuration:
Teensy 4.0 with nothing attached except USB for download and serial monitor
Latest versions of VS Code and PlatformIO (latest version of Teensy40)
Teensy Loader 1.52

I'm using the string functions four function calls deep from setup(). To isolate the problem, I commented out most of my actual code and use the following as a test:

char *p;
p = strstr("Test String", "str");

I used literal strings to eliminate the possibility I was passing in a NULL pointer. This always causes a crash, but when I comment it out and put the same code into any of the three higher-level calling functions...no crash.

I thought it might be a stack overflow problem, since I had been using a 300-byte buffer as a local variable in the one of the three calling functions. I haven't yet been able to find the default stack allocation, but I saw some clues that it might be quite large. Anyway, I changed the buffer to a global and that didn't have any effect on the problem.

I'll keep plugging away at this (and maybe I'll just write my own version of strstr()), but I thought it was time to run the flag up for some help. Thanks!

[fab672000]

Stack and heap memory usually share in the same ram memory space on microcontrollers (not talking about additional spi ram/flash memory here), so allocating data on the heap would still not convince me that you don't have a memory overflow?

Two things I would try:
1. Reduce this global buffer size memory to 15 bytes and try again.
2. Check about the validity / volatility of the buffer char* pointer (p in your example) that you return (btw it would be cleaner to return a const char* if you don't need to modify it ),
sometimes you just return a bad (other uninitialized, non-zero terminated pointer) ; can't see that from your code so pardon my naive suggestion.

[manitou]

can you provide a complete simple sketch that demonstrates the problem?? Can you try your test with the Arduino/Teensy IDE ?

[Aerokeith]

I created a completely separate program to test strstr() when called several levels deep (see below) and of course, it works fine (ugh!). I included a 300-byte local variable buffer at each level to try to simulate a "large" stack. So it must be something about the particular code that I was calling it from, although it's weird that it works fine without the one line that calls strstr() and then crashes when I uncomment the line. So I'll keep testing to try to get more clues. If you're curious, below is the separate test code I created.

#include <Arduino.h>

void Level1();
void Level2();
void Level3();
void Level4();

void setup() {
  char *p;
  char dummy[300];

  Serial.begin(115200);
  delay(3000);
  p = strstr("Test string 0", "str");
  if (p != NULL) {
    Serial.print("Level 0: ");
    Serial.println(p);
  }
  Level1();
}

void Level1() {
  char *p;
  char dummy[300];
  p = strstr("Test string 1", "str");
  if (p != NULL) {
    Serial.print("Level 1: ");
    Serial.println(p);
  }
  Level2();
}

void Level2() {
  char *p;
  char dummy[300];
  p = strstr("Test string 2", "str");
  if (p != NULL) {
    Serial.print("Level 2: ");
    Serial.println(p);
  }
  Level3();
}

void Level3() {
  char *p;
  char dummy[300];
  p = strstr("Test string 3", "str");
  if (p != NULL) {
    Serial.print("Level 3: ");
    Serial.println(p);
  }
  Level4();
}

void Level4() {
  char *p;
  char dummy[300];
  p = strstr("Test string 4", "str");
  if (p != NULL) {
    Serial.print("Level 4: ");
    Serial.println(p);
  }
}

void loop() {
  // put your main code here, to run repeatedly:
}

[el_supremo]

Your sample code in msg #4 is not the same as the snippet in #1.

p = strstr("Test String", "str");

will return NULL because "str" does not occur in "Test String".
Your tests in #4 all use "Test string", in which case it will match "str".
Is your code handling a NULL return from strstr properly?

Pete

Pete

[fab672000]

Apart from the fact that this example will not return NULL as you test with 'string' not 'String', and assuming that:
1. you handle the NULL case properly in your original code
2. if you return a pointer in your original, that the pointer is referencing a valid zero terminated string,
then you may have a memory overflow, did you test with a very small buffer size as I recommended?
Also what does the Arduino compiler output tell you after compiling (your original code!), it should tell you exactly how much ram percentage you code is currently using ; which should give you an idea of the stack left for the rest.

[Aerokeith]

Sorry, I manually (mis-)typed the code in the original snippet; the actual test code used "Test string" not "Test String". The program produced the following (correct) output:

Level 0: string 0
Level 1: string 1
Level 2: string 2
Level 3: string 3
Level 4: string 4

The actual code (not the test program) is only using 15.8% of RAM, so it doesn't seem to be a memory overflow problem. Changing the buffer size didn't affect the results.
I went back to the original code and commented out almost everything, but left in place the 4-deep function calls. I was passing some pointers-to-pointers as arguments, so I temporarily made these globals to eliminate that as a possible issue. Now I'm in the process of incrementally uncommenting (adding back in) individual lines to see when it crashes. As of right now, I'm able to execute strstr() without crashing, but it IS crashing when I assign a non-null structure pointer value to a same-type pointer that was null. That is:

scriptRec *scriptHead;
scriptRec *scriptTail;

if ((scriptTail == NULL) && (scriptHead != NULL)) {
    Serial.println("ScriptTail == NULL");
    scriptTail = scriptHead;
  }

This consistently crashes when I uncomment the pointer assignment line (and stops crashing when I comment it out). Very strange.

[fab672000]

Typically when you get to this kind of trouble the problem lies somewhere else in your code that where you currently focus:
the pointer assignment is obviously not the root cause of the crash,
but I can see that these pointers are not assigned to NULL before used, so it may simply crash because when you set scriptTail to a non null (yet uninitialized or freed) pointer value,
the rest of the code will now use this pointer and crash, because the code flow is different.

[fab672000]

What I always recommend to beginners especially in c/c++ languages is to _always_ initialize the variables they create

scriptRec *scriptHead = NULL;
scriptRec *scriptTail = NULL;

, this is because c/c++ languages don't guaranty that memory is going to be zeroe'd before used, it depends on the platform and compilers and most of the time they don't. This is because of performances reasons...
By default, and even after many years of development ; I still do that and only optimize what needs to be optimized, read (as an example) : http://www.moscowcoffeereview.com/pr...-optimization/

Contrastingly, in .net languages like c sharp ; variables/attributes are always initialized to their default values (usually 0/null/false depending on their type).

Another track of investigation if you can't provide your full code is to check for out of range access (i.e. an array / struct is written out of range, which would certainly cause trouble including crashes too).

[Aerokeith]

Thanks for all the suggestions. I think I have a handle on this now. The thing that threw me was that crashes prevent the output of serial monitor messages that were generated BEFORE the offending instruction. This makes sense assuming that serial output is interrupt-driven from a TX buffer. But it sure made it hard to pinpoint the problem (reference to a null pointer) that was WAY, WAY after the section that I was incorrectly focusing on. Although it seemed like my comment-out/in approach seemed to isolate the problem, I think there must have been some side-effects on the subsequent code that actually caused the crash. Sure wish I was able to use a debugger!
@fab672000, although this didn't make a difference here, I'll take your advice to always initialize pointers to NULL, just to be sure!

By the way, the way I got around the serial monitor issue described above was to simply add a delay(3000) after every Serial.print statement added for debugging.

[jonr]

I believe that Serial.flush() after a print is the best thing to use for debugging.

[Aerokeith]

Ah, yes! I had meant to go looking for something like that (which I assumed must exist) but I never got around to it. Thanks for the reminder!

[defragster]

There is a software debugger interface thread :: GDB ???

It will break on a fault and ideally leave a stack trace back to(ward) the point of offense.

[Aerokeith]

Thanks! I scanned the thread and will read more carefully later. Not sure if it only works with Visual Micro (and I'm not really sure what that is).

[defragster]

Visual Micro not required, may be integrated there - but not related.

It uses DUAL USB on Teensy and existing GDB CMDline tool already installed with the library on the thread.

I've tested on Windows and integrated startup in build using the TSET from editor ... or CMDline to build with Arduino IDE and TyCommander. It has tool to integrate to IDE as well - but I never tried that.

[defragster]

This thread : Using-GDB-with-Teensy-without-hardware-debugger-first-Beta

For this Library: github.com/ftrias/TeensyDebug