Questions Surrounding Space Saving Options in Code

anatoledp

Active member
Ive been working on making a scripting language work on the teensy. it however does not completely fit into ITCM memory space. Ive started to go through and mark areas that is not directly tied to the runtime to be executed from flash via the FLASHMEM declaration. I have a question now about the strings. I see that wrapping strings in F() will store them and access them from flash but how does this relate to functions marked by FLASHMEM? like for instance
Code:
FLASHMEM void somefunc ()
{
    //do something
    specialFunc("random string");
    char* otherString = "another string";
    //do other things
}
would wrapping the "random string" and "another string" do anything or will it already be in flash because the function itself is in flash? Also though the function itself is in flash are there things in a function that are not stored in there? like any type of variables or anything that is still put into itcm regardless of the FLASHMEM declaration?
 
That thread is bad advice because it no longer applies. Constant strings are stored in flash, but they also get copied into memory (and hence take up space) at startup by default.
On the Teensy 4.x the F() macro does nothing - forget about using it. Instead use PSTR() to wrap any string literals. And yes, you need to do it for any literals regardless of the function they're declared in; FLASHMEM only applies to the actual code generated for the function, not the data used within it.
For any constant data that isn't string literals, you should use the PROGMEM specifier e.g. "static const uint32_t some_array[3] PROGMEM = {0, 1, 2};"
 
Hmmm. It appears you are (once again) correct.

I have code like this in one of my project files:
Code:
static const char * const distanceUnits[] PROGMEM = {   "km", "miles"   };
pgmspace.h has this:
Code:
#define PSTR(str) ({static const char data[] PROGMEM = (str); &data[0];})
So I tried to add the following:
Code:
return PSTR("UltimateGPS v3");
Which resulted in the following familiar error:
Code:
/home/skunk/.platformio/packages/framework-arduinoteensy/cores/teensy4/avr/pgmspace.h:35:39: error: 'data' causes a section type conflict with 'distanceUnits'
 
Yeah PSTR has a habit of triggering that error quite often. I don't fully understand why, suspect it's some c++/compiler bug. It's really unhelpful that GCC won't actually state what the conflict is between the different section declarations, because then it would be obvious which one was incorrect.
 
The FLASHMEM and PROGMEM conflict ... Code in FLASH won't link with DATA in Flash giving the: "causes a section type conflict"?

This works _ but only to change (duplicate?) DATA in FLASH? RAM1 DATA same in both?:
Code:
void setup() {
  while (!Serial)
    ;  // wait
  somefunc();
  somefuncB();

  Serial.println("\nPart two ------");
  Serial.printf("\nCALL: %s\n", somefunc() );
  Serial.printf("\nCALL_B: %s\n", somefuncB() );
}

void loop() {
}

//#define boop PSTR
#define boop

FLASHMEM const char * somefunc ()
{
  Serial.printf("SF:\t%s\n", boop("UltimateGPS v3 >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< ") );
  return boop("UltimateGPS v3 >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< ");
}

FLASHMEM const char * somefuncB ()
{
  const char* otherString = boop("another string >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< ");
  Serial.printf("SF_B:\t%s\n", otherString );
  return otherString;
}

Below with :#define boop
Code:
SF:    UltimateGPS v3 >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**<
SF_B:    another string >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**<

Part two ------
SF:    UltimateGPS v3 >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**<

CALL: UltimateGPS v3 >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**<
SF_B:    another string >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**<

CALL_B: another string >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**<

SIZE with :#define boop
Code:
Memory Usage on Teensy 4.1:
  FLASH: code:35460, data:5064, headers:8624   free for files:8077316
   RAM1: variables:5920, code:32768, padding:0   free for local variables:485600
   RAM2: variables:12416  free for malloc/new:511872

And same output and the SIZE with: #define boop PSTR
Code:
Memory Usage on Teensy 4.1:
  FLASH: code:35460, data:5508, headers:9204   free for files:8076292
   RAM1: variables:5920, code:32768, padding:0   free for local variables:485600
   RAM2: variables:12416  free for malloc/new:511872
 
That thread is bad advice because it no longer applies. Constant strings are stored in flash, but they also get copied into memory (and hence take up space) at startup by default.
On the Teensy 4.x the F() macro does nothing - forget about using it. Instead use PSTR() to wrap any string literals. And yes, you need to do it for any literals regardless of the function they're declared in; FLASHMEM only applies to the actual code generated for the function, not the data used within it.
For any constant data that isn't string literals, you should use the PROGMEM specifier e.g. "static const uint32_t some_array[3] PROGMEM = {0, 1, 2};"
maybe im missing something. Lets take a direct example out of my code . . .
C++:
// interface
FLASHMEM int asCScriptEngine::RegisterGlobalFunction(const char *declaration, const asSFuncPtr &funcPointer, asDWORD callConv, void *auxiliary)
{
#ifdef AS_MAX_PORTABILITY
    if( callConv != asCALL_GENERIC )
        return ConfigError(asNOT_SUPPORTED, PSTR("RegisterGlobalFunction"), declaration, 0);
#endif

    asSSystemFunctionInterface internal;
    int r = DetectCallingConvention(false, funcPointer, callConv, auxiliary, &internal);
    if( r < 0 )
        return ConfigError(r, PSTR("RegisterGlobalFunction"), declaration, 0);

    isPrepared = false; // TODO: Only set this after the validations have been completed, to avoid unnecessary Prepare in case no change was made

    // Put the system function in the list of system functions
    asSSystemFunctionInterface *newInterface = asNEW(asSSystemFunctionInterface)(internal);
    if( newInterface == 0 )
        return ConfigError(asOUT_OF_MEMORY, PSTR("RegisterGlobalFunction"), declaration, 0);

    asCScriptFunction *func = asNEW(asCScriptFunction)(this, 0, asFUNC_SYSTEM);
    if( func == 0 )
    {
        asDELETE(newInterface, asSSystemFunctionInterface);
        return ConfigError(asOUT_OF_MEMORY, PSTR("RegisterGlobalFunction"), declaration, 0);
    }

    func->sysFuncIntf = newInterface;

    asCBuilder bld(this, 0);
    r = bld.ParseFunctionDeclaration(0, declaration, func, true, &newInterface->paramAutoHandles, &newInterface->returnAutoHandle, defaultNamespace);
    if( r < 0 )
    {
        // Set as dummy function before deleting
        func->funcType = asFUNC_DUMMY;
        asDELETE(func,asCScriptFunction);
        return ConfigError(asINVALID_DECLARATION, PSTR("RegisterGlobalFunction"), declaration, 0);
    }

    // TODO: namespace: What if the declaration defined an explicit namespace?
    func->nameSpace = defaultNamespace;

    // Check name conflicts
    r = bld.CheckNameConflict(func->name.AddressOf(), 0, 0, defaultNamespace, false, false, false);
    if( r < 0 )
    {
        // Set as dummy function before deleting
        func->funcType = asFUNC_DUMMY;
        asDELETE(func,asCScriptFunction);
        return ConfigError(asNAME_TAKEN, PSTR("RegisterGlobalFunction"), declaration, 0);
    }
    
    // Validate property signature
    if( func->IsProperty() && (r = bld.ValidateVirtualProperty(func)) < 0 )
    {
        // Set as dummy function before deleting
        func->funcType = asFUNC_DUMMY;
        asDELETE(func,asCScriptFunction);
        if( r == -5 )
            return ConfigError(asNAME_TAKEN, PSTR("RegisterGlobalFunction"), declaration, 0);
        else
            return ConfigError(asINVALID_DECLARATION, PSTR("RegisterGlobalFunction"), declaration, 0);
    }

    // Make sure the function is not identical to a previously registered function
    asUINT n;
    const asCArray<unsigned int> &idxs = registeredGlobalFuncs.GetIndexes(func->nameSpace, func->name);
    for( n = 0; n < idxs.GetLength(); n++ )
    {
        asCScriptFunction *f = registeredGlobalFuncs.Get(idxs[n]);
        if( f->IsSignatureExceptNameAndReturnTypeEqual(func) )
        {
            func->funcType = asFUNC_DUMMY;
            asDELETE(func,asCScriptFunction);
            return ConfigError(asALREADY_REGISTERED, PSTR("RegisterGlobalFunction"), declaration, 0);
        }
    }

    func->id = GetNextScriptFunctionId();
    AddScriptFunction(func);

    currentGroup->scriptFunctions.PushLast(func);
    func->accessMask = defaultAccessMask;
    registeredGlobalFuncs.Put(func);

    // If parameter type from other groups are used, add references
    currentGroup->AddReferencesForFunc(this, func);

    // Return the function id as success
    return func->id;
}
without the PSTR i get this for the memory showcase
Code:
teensy_size: Memory Usage on Teensy 4.1:
teensy_size:   FLASH: code:500360, data:103360, headers:8628   free for files:7514116
teensy_size:    RAM1: variables:109888, code:337672, padding:22776   free for local variables:53952
teensy_size:    RAM2: variables:12416  free for malloc/new:511872
and with PSTR i get this . . .
Code:
teensy_size: Memory Usage on Teensy 4.1:
teensy_size:   FLASH: code:500408, data:103568, headers:8372   free for files:7514116
teensy_size:    RAM1: variables:109888, code:337672, padding:22776   free for local variables:53952
teensy_size:    RAM2: variables:12416  free for malloc/new:511872
This is compiled using TEENSY_OPT_SMALLEST_CODE_LTO if that plays any factor in this.
but as u can see only the flash data has any change (increase in what i am assuming is the string placement) . . . the RAM1 stuff still remains exactly the same . . . is there something that needs to happen or in senarios like this? There are tons of strings scattered around the code that is in FLASHMEM declared functions that would be helpful to decrease size so more code can be stored in ITCM without crossing the 32k boundary
 
I have no idea what your ConfigError function does so no context for how the strings are used. Maybe they're ignored completely and the compiler is discarding them. Otherwise the strings appear to all be identical so the compiler would be combining them into a single occurrence; using PSTR would disable that, because they are no longer be processed as literals by the optimizer.
This is why it's important to examine the program map or listing before attempting to optimize.
 
Maybe that was a bad example because it has same strings throughout. I was just showcasing it in particular since it was one of the smaller ones that don't extend 50+ loc. But it's pretty much if not the same case throughout. On larger portions that have different strings it even increases code size in itcm as well as flash (variables is iffy but never a decrease, only increase) hence the questions surrounding strings.
 
Post #6 example code showed the same result!

No change in RAM1 usage, but:
data:5064 .vs. data:5508 and data:103360 .vs. data:103568

It looks like PSTR just places a Duplicate copy in Flash - but another copy also placed in the area downloaded to RAM1 for 'use'.

When that fails is perhaps when the "causes a section type conflict" occurs because the FLASH copy not accessible to the code expecting to address it in RAM1?

P#6 is a simple example with mulitple uses [ with or without PSTR ] present that should be easy to modify for effect and test with change to these two lines:
Code:
//#define boop PSTR
#define boop
Or other edits as needed to see the desired effect as this as written is not giving the expected/desired result.
 
It has the same issue, you are using an identical string in multiple places which would be replaced with one copy by the compiler if and only if the string is treated like a proper string literal and not wrapped with PSTR.
 
It has the same issue
Correct, I tried to recreate the issue in a short clear sketch that compiled without the Odd Error and allowed for with and without PSTR testing.

Results below look to show improvement using PSTR when the three strings were edited with "One, Two, Three" in following code. Comparing to results above it seems the output run results may have been swapped, except the "headers" size - these are not. Never 'saw'/observed "headers" before in FLASH - it grows with PSTR. It is grown by 568 bytes the same as non-PSTR has larger data in FLASH. Three strings now add to 470 bytes.

Another oddity is the RAM1 again has no padding and code 32768 and then DATA of 5-6KB not taking another 32KB chunk. So that math is off.

Memory Usage on Teensy 4.1: WITHOUT PSTR
FLASH: code:35460, data:6088, headers:8624 free for files:8076292
RAM1: variables:6944, code:32768, padding:0 free for local variables:484576
RAM2: variables:12416 free for malloc/new:511872

Memory Usage on Teensy 4.1: WITH PSTR
FLASH: code:35460, data:5520, headers:9192 free for files:8076292
RAM1: variables:5920, code:32768, padding:0 free for local variables:485600
RAM2: variables:12416 free for malloc/new:511872

Code:
void setup() {
  while (!Serial)    ;  // wait
  somefunc();
  somefuncB();

  Serial.println("\nPart two ------");
  Serial.printf("\nCALL: %s\n", somefunc() );
  Serial.printf("\nCALL_B: %s\n", somefuncB() );
}

void loop() { }

#define boop PSTR
//#define boop

FLASHMEM const char * somefunc ()
{
  Serial.printf("SF:\t%s\n", boop("UltimateGPS v3 ONE >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< ") );
  return boop("UltimateGPS v3 TWO >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< ");
}

FLASHMEM const char * somefuncB ()
{
  const char* otherString = boop("another string THREE >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< >> abcdefghijklmnopqrstuvwxyz0123456789**< ");
  Serial.printf("SF_B:\t%s\n", otherString );
  return otherString;
}
 
Another oddity is the RAM1 again has no padding and code 32768 and then DATA of 5-6KB not taking another 32KB chunk. So that math is off.
I don't see how it is wrong. RAM1 is divided into ITCM and DTCM. The size used for ITCM is the same for both builds (32KB), but when PSTR is used there is less allocated data in DTCM so more space remains for local variables (stack).
 
Misread it seems - but odd the code is perfectly filling the 32KB - it hasn't changed since p#6 - but odd the exact value was hit.

So is PSTR() working as expected ? Any edits to example that would show other uses or value?
 
Back
Top