Is using a "String" still a bad idea?

Status
Not open for further replies.

bvernham

Well-known member
I have already been told, and read that using a "String" instead of a "string" (char array) in a microcontroller is a bad idea due the indeterminant amount of memory that the "String" will occupy.

Is this still the case or has some magic occur in the compiler to make "String" a safe alternative to use in microcontroller code?

The reason why I ask the question is that because I just started working with bluetooth with the aim of using for a Android based GUI/HMI for the CAN logger I have for the Teensy4.1.

I was looking at the Sparkfun HM-1xx library and guess what, "String" declarations. Puzzled I asked Sparkfun if this was "OK".

So then I was looking at the various Android apps and thought Virtuino looked interesting. I opened the VirtuinoCM library and guess what, "String" declarations.

There is a positng in the forum about using Virtuino and creating your own library for instead of VirtuinoCM and the poster has "String" declarations.

Did I miss the memo in which using "String" in microcontrollers at will is now a safe and will not cause any potential instability/intermittent bugs?

If so when/how did this occur?

Thanks

Bruce
 
Strings (core code) seem to have proper support to function well and good in general on Teensy ... as far as known and tested ... YMMV.

sz_C strings are not risk free - but don't dynamically play games w/memory.

As long as memory fits and functions
 
Opinions vary regarding use of malloc / free and dangers of possible memory fragmentation.

Some people always draw a very hard line and insist that no dynamic memory allocation is ever used in any way. If you're developing critical products like medical devices, aircraft controls, military weapons systems, you probably should be extremely cautious. Lives are on the line for those applications!

For less critical applications, many people feel a more pragmatic approach of observing the actual memory usage under the heaviest anticipated use cases and allowing a safety margin of extra memory is good enough. While you can't prove in a mathematically certain way the worst case, you can estimate the odds. For many cases where you use much less than the total memory, it's easy to show the odds are very low, in many cases pretty much virtually zero. Usually for those sorts of applications, you accept a tiny chance of failure (eg, also no redundant power supplies, battery backups, other ways things can go wrong) and generally try to design the system to be able to recover as gracefully as possible if a problem does occur. Sometimes just showing the user clear feedback about a problem is most important.

When using String, you can mitigate much of the risk by using the reserve(size); function on "long life" String variables. Especially for a String which stay around and gradually grows in size as you collect more data, reserving the maximum size up front greatly reduces the chances of memory fragmentation problems.

Which way you choose really depends on a judgement call where you balance the consequence of a failure, an estimate of the risk, and the extra cost incurred. It's an easy answer for people on the internet to say everyone else should never take any risk, no matter how unlikely, even when the consequence isn't severe, and even if the cost of guarding against it is high. That is always the safe answer. It's an answer that feel virtuous. Whether that sort of answer really makes sense in the practical reality of your application is call you have to make.


Puzzled I asked Sparkfun if this was "OK".

I'm curious to hear what Sparkfun says (if they reply). Hope you'll share.
 
Thanks for the reply's. I have asked both Sparkfun and Virtuino and no responses. Frankly this forum is the first place I have seen a response of any type so kudos.

I am just trying to wrap my head around why use "String" instead of "string". Yes I know some of the handling of "String" is easier like for example "+" instead of "concat" but for example in this code snipit:
Code:
 String get() {
      String returnLine = "";

      byte inByte;
      
      while ( Bluetooth.available() ) {
       
         inByte = Bluetooth.read(); 
         switch (inByte) {

            case '\n':   // end of text
               input_line [input_pos] = 0;  // terminating null byte

               // terminator reached! now process input_line
               // reset buffer for next time
               input_pos = 0; 
               returnLine = input_line;
               return returnLine; 
               break;
            case '\r':   // discard carriage return
               break;
            case '!':    // start of virtuino command
               input_pos = 0;
               input_line [input_pos++] = inByte;
               break;
            case '$':    // end of virtuino command 
               input_line [input_pos++] = inByte;
               input_line [input_pos++] = 0;  // terminating null byte
Yes, I know this is is not complete but just illustrative. All the handling of the incoming data is done with a "string" and even formatted with the '\0' terminator but then returned as a "String" instead of a "string"? Why?

I am not understanding the any real or perceived advantages of "String" except it makes things "easier". But I do not understand how having two forms of the same values (the text) represented in two different ways "string" and "String" is easier.

Just and observations from someone with only moderate coding skills.

Thanks

Bruce
 
+1 on what Paul said. For some cases, the MISRA-C policy of "Dynamic heap memory allocation shall not be used" is best.

> the odds are very low,

But note that if there is any chance of someone trying to deliberately cause a problem, the odds go way up.
 
For platforms other than Teensy, I sometimes use an intermediate policy of using malloc() to do one-time allocations of variables that exist "forever", such as communication buffers, but never use free(). There are two reasons I might do this instead of using static variable allocations. One is that I want the variable to be in the RAM assigned to heap/stack because it is faster (or slower) than the RAM used for static variables. The other reason is that buffer sizes can be arguments to configuration functions.
 
What is the proper way to detect an out-of-memory failure of "new" or some std::string related statement (eg, S += "x") in a teensy program?
 
I did some tests:

Looks like "new" with no memory will return NULL which is non-conforming C++.

String operations that can't allocate more memory generally quietly fail, with who knows what effect on your program. In some cases, it may crash/reset.

The standard solution is exceptions, but it's not clear that these work on teensy.

I didn't see errno change.

With what I know (not much), if there are concerns about lack of memory and wrong program behavior, avoid Strings on a teensy.
 
Looks like "new" with no memory will return NULL which is non-conforming C++.

The standard solution is exceptions, but it's not clear that these work on teensy.

C++ exceptions and RTTI are not supported on Teensy. All C++ code is compiled with flags "-fno-exceptions -fpermissive -fno-rtti".


String operations that can't allocate more memory generally quietly fail, with who knows what effect on your program.

String it supposed to drop data when it can't allocate memory, so you get a proper String variable which is blank.


In some cases, it may crash/reset.

Can you demonstrate any of those cases?
 
> String it supposed to drop data when it can't allocate memory, so you get a proper String variable which is blank.

Here is code that eventually drops data but doesn't result in a String variable that is blank. Then much later it crashes (yes, checking S2 for NULL would probably solve this). Teensy 4.0, 1.8.15, debug optimization.

Code:
void setup() {
  // put your setup code here, to run once:

  delay(2000);

  Serial.println("hello");
  
}

String S = "abc";

  
void loop() {
 
  S += S;

  String *S2 = new String;

  *S2 = "x2--------------------------------------------------------------------------------------------------------------------";
  
  Serial.printf("%d\n", S.length());
  Serial.println(*S2);

  delay(5);

}
 
Last edited:
I'm confused ?!? What do you expect that this code should do? You are allocating memory in loop without ever releasing it and not checking if the allocation was successful. Since the Teensy doesn't have an infinite amount of memory this has to crash finally. And, of course this has nothing to do with String. It wouldcrash for every other type as well.

Here some code with proper checking which doesn't crash
Code:
#include "Arduino.h"
#include <cstdlib>
#include <new>

void setup()
{
    while (!Serial){}

    if(CrashReport)
        Serial.println(CrashReport);
    // put your setup code here, to run once:

    Serial.println("hello");
}

String S = "abc";

void loop()
{
   S += S;

   String* S2 = new (std::nothrow) String();
   if (S2)
   {
        *S2 = "x2--------------------------------------------------------------------------------------------------------------------";
        Serial.println(*S2);
    }
    else
    {
       Serial.println("Out of memory");
       while (1) {}
    }
    Serial.printf("%d\n", S.length());

    Serial.println(millis());

    delay(10);
}

After some 20s it runs out of memory, detects this and gracefully stops...

Code:
x2--------------------------------------------------------------------------------------------------------------------
196608
21563
x2--------------------------------------------------------------------------------------------------------------------
196608
21573
x2--------------------------------------------------------------------------------------------------------------------
196608
21583
Out of memory

new usually throws if it can't alllocate memory which won't work here. You have to use new (std::nothrow) if you wan't it to return a nullptr in stead of trying to throw a bad alloc exception.
 
new usually throws if it can't alllocate memory which won't work here. You have to use new (std::nothrow) if you want it to return a nullptr in stead of trying to throw a bad alloc exception.

@luni, good to know! I tried a similar test, but without std::nothrow, and found that S2 was never NULL.
 
Yes this can be confusing. Maybe it is possible to change the default behavior somehow? Need to goolgle this on a rainy Sunday afternoon...
 
> you can mitigate much of the risk by using the reserve(size)

This leads to how does one safely use reserve()?

> reserve(): A bad_alloc exception is thrown if the function needs to allocate storage and fails.

But "C++ exceptions and RTTI are not supported on Teensy".

I suggest that it's still true: reliability will typically decrease as you proceed along "static->stack->malloc()->String" for string memory allocation on a teensy.
 
This leads to how does one safely use reserve()?

Reserve returns false if it can't reserve. So just check the return value...

reserve(): A bad_alloc exception is thrown if the function needs to allocate storage and fails.

No, reserve uses malloc which can not throw. https://github.com/PaulStoffregen/c...c4fa80ad6c440fe54110/teensy4/WString.cpp#L130

I would be really interested in an example where the correct usage of Strings crashes a Teensy. I was trying hard but couldn't crash it so far. Here some tests using reserve:

Code:
#include "Arduino.h"

void panic(String err)
{
    Serial.println(err);
    //while (true) yield();
}

void testStack()
{
    // try to geneate 3 200kB strings on the stack.
    // storage will still be on the heap
    // The third trial should run out of memory
    Serial.println("Test Stack");
    String s1, s2, s3;
    if (!s1.reserve(200 * 1024)) panic(" Out of Memory s1");
    if (!s2.reserve(200 * 1024)) panic(" Out of Memory s2");
    if (!s3.reserve(200 * 1024)) panic(" Out of Memory s3");

    // strings go out of scope here and release the allocated memory
}

void testHeap()
{
    // try to geneate 3 200kB strings on the heap.
    // storage will also be on the heap
    // The third trial should run out of memory

    Serial.println("Test Heap");
    String *s1 = new String();
    String *s2 = new String();
    String *s3 = new String();

    if (!s1->reserve(200 * 1024)) panic(" Out of Memory *s1");
    if (!s2->reserve(200 * 1024)) panic(" Out of Memory *s2");
    if (!s3->reserve(200 * 1024)) panic(" Out of Memory *s3");

    delete (s1);
    delete (s2);
    delete (s3);
}

void testRandom()
{
    Serial.println("Test Fragmentation");

    constexpr int arrSize = 10;
    constexpr int min     = 1 * 1024;
    constexpr int max     = 100 * 1024;

    for (int loop = 0; loop < 100'000; loop++)
    {
        String strings[arrSize];
        for (int i = 0; i < arrSize; i++)
        {
            size_t strSize = random(min, max);
            String &s      = strings[i];

            s = "loop: " + String(loop) + " " + String(i) + " requested: " + String(strSize);
            if (s.reserve(strSize))
            {
                s += " OK";
            } else
            {
                 s += " not enough memory";
            }
            Serial.println(strings[i]);
            delayMicroseconds(50); 
        }
    }

     Serial.println("Test Fragmentation done");
}

void setup()
{
    while (!Serial) {}

    if (CrashReport)
        ;

    testStack();
    testHeap();

    testStack();
    testHeap();

    testRandom();
}

void loop()
{
}

Prints:
Code:
Test Stack
 Out of Memory s3
Test Heap
 Out of Memory *s3
Test Stack
 Out of Memory s3
Test Heap
 Out of Memory *s3
Test Fragmentation
loop: 0 0 requested: 64066 OK
loop: 0 1 requested: 34555 OK
loop: 0 2 requested: 17823 OK
loop: 0 3 requested: 75488 OK
loop: 0 4 requested: 59225 OK
loop: 0 5 requested: 77953 OK
loop: 0 6 requested: 28250 OK
loop: 0 7 requested: 43649 OK
loop: 0 8 requested: 36198 OK
loop: 0 9 requested: 33185 OK
loop: 1 0 requested: 46862 OK
loop: 1 1 requested: 27996 OK
loop: 1 2 requested: 36174 OK
loop: 1 3 requested: 15028 OK
loop: 1 4 requested: 25306 OK
loop: 1 5 requested: 21740 OK
loop: 1 6 requested: 60975 OK
loop: 1 7 requested: 22517 OK
loop: 1 8 requested: 95584 OK
loop: 1 9 requested: 91006 OK
loop: 2 0 requested: 34954 OK
loop: 2 1 requested: 96749 OK
loop: 2 2 requested: 93755 OK
loop: 2 3 requested: 101066 OK
loop: 2 4 requested: 60715 OK
loop: 2 5 requested: 34517 OK
loop: 2 6 requested: 63874 OK
loop: 2 7 requested: 3689 OK
loop: 2 8 requested: 32029 not enough memory
loop: 2 9 requested: 94147 not enough memory
loop: 3 0 requested: 19076 OK
loop: 3 1 requested: 88083 OK
loop: 3 2 requested: 59918 OK
loop: 3 3 requested: 81062 OK
....
 
Thanks for the info.

So one shouldn't expect teensy Strings to act like a C++ std::string or according to Arduino documentation (where we find "reserve() Returns Nothing").

@bvernham: this adds additional risks to using String in teensy code.
 
Status
Not open for further replies.
Back
Top