Disabling firmware size limit / error on exceeding size

doctea

Member
Hullo,

I'm trying to use the 'Inspect' feature of platformio to investigate memory use so that I can find the best places to start optimising my (apparently very large) program.

However, my project is already so large that when 'Inspect' compiles its debug version of the firmware, it exceeds the size limit, which throws an error and halts the 'Inspect' process -- just when I needed it the most!

Is there a way to disable the 'fail on excessive size' (teensy_size?) check when using 'Inspect'?

Cheers!
 
Does PIO run PJRC's Teensy_Size? : "teensy_size: usage: teensy_size <file.elf>"

That monitors and displays the space usage for Arduino builds
 
What would be the point of disabling the check? The program still wouldn't fit in the flash memory.
The "inspect" feature shows a breakdown of how much every variables/functions/classes/files/library is taking in the binary, so even without being able to run it I'll be a able to see where to target my size optimisations.

By disabling certain features I'm able to get it small enough to upload and run -- I might be wrong but I believe that by design "inspect" uses different compiler options in order to produce that breakdown, which in my case seems to have the side effect of increasing the size enough for the inspection attempt to fail..
 
Does PIO run PJRC's Teensy_Size? : "teensy_size: usage: teensy_size <file.elf>"

That monitors and displays the space usage for Arduino builds
Yeah, it looks like it does, and I'm wondering if maybe even the build process itself is only failing because teensy_size is returning non -zero. I've had a bit of a poke around but I can't seem to figure out where it's configured in pio and how to adjust that. Not sure where to look next to figure it out, hence the post here :)

I was going to try replacing teensy_size with modified version to add a switch to ignore failures but also not sure what environment I need to set up to generate a suitable exe?
 
I might be wrong but I believe that by design "inspect" uses different compiler options in order to produce that breakdown, which in my case seems to have the side effect of increasing the size enough for the inspection attempt to fail..
In that case the information it gives wouldn't be worth anything, since different compiler options can greatly influence what gets included in the final build vs what gets optimized away. Even the way structures are packed can be changed by simple compiler options.

You're better off just doing a regular build and examining the .map file.
 
In that case the information it gives wouldn't be worth anything, since different compiler options can greatly influence what gets included in the final build vs what gets optimized away. Even the way structures are packed can be changed by simple compiler options.

You're better off just doing a regular build and examining the .map file.
Hmm. I'll look into this, thanks. I was under the impression though that to get useful information about what's what in the binary that compiler optimisations had to be disabled and debug symbols included, so this 'rough approximation' of sizes given by the 'Inspect' tool was the best thing available. Otherwise, why doesn't the Inspect tool just do it that way anyway?
 
Source code for teensy_size:

Thanks, I had seen this but it wasn't clear whether returning 0 from this would allow the 'Inspect' task to complete without aborting.

Only way to find out is probably to try it I guess, but what sort of environment do I need to set up in order to compile a new teensy_size.exe on Windows? I'm fairly used to compiling stuff on *nix and via vscode/platformio but no idea what's involved in compiling new teensy toolchain stuff.

Are there some docs on this anywhere?
 
The cmdline details are in p#3 - teensy_size can be manually executed against the ELF file to see what it shows.

As p#6 noted the source for that is on github
 
The cmdline details are in p#3 - teensy_size can be manually executed against the ELF file to see what it shows.

As p#6 noted the source for that is on github
I'm obviously not communicating this very well.

The teensy_size output isn't what I'm seeking. I'm seeking the variable -by- variable, function -by-function, file -by-file breakdown that the platformio "inspect" function usually shows, but which is unable to complete when the project is already near to the firmware size limit.

Are there docs anywhere on what environment I need to build teensy_size from source?
 
I'm seeking the variable -by- variable, function -by-function, file -by-file breakdown that the platformio "inspect" function usually shows

This question might be better asked on PlatformIO's issue tracker. They'll probably need you to share the large code that gives the error.


Are there docs anywhere on what environment I need to build teensy_size from source?

It's just a couple .c files which don't depend on any special libraries, just a C compiler and the usual standard C library that comes with it. The makefile should work on any system where "gcc" and "make" commands are installed. Most Linux systems have this by default, or you can install it with "apt install build-essential". On MacOS, you would need to install the Xcode command line package which Apple publishes. I don't know what you would do to get these on Windows, as I really don't use Windows much. On Linux, if you install mingw and edit the makefile so gcc is replaced with x86_64-w64-mingw32-gcc, running "make" will cross compile it. Just rename the compiled teensy_size file to have a .exe extension and copy to any Windows machine to run it.
 
I'm obviously not communicating this very well.

The teensy_size output isn't what I'm seeking. I'm seeking the variable -by- variable, function -by-function, file -by-file breakdown that the platformio "inspect" function usually shows, but which is unable to complete when the project is already near to the firmware size limit.

Are there docs anywhere on what environment I need to build teensy_size from source?
The PlatformIO inspect function does the analysis on a compiled ELF file, compiled in debug mode (to get extra info to generate the report). Obviously, the linker cannot generate the ELF if section sizes are too large for the regions. So, it's not exactly a PlatformIO issue.....

Does it build normally, outside of the PlatformIO inspect function, either release or debug builds? It if does, what is the reported size from that build? There are several sections involved, any one of them being exceeded will prevent the linker, and knowing which section overflows is a clue to where your issue is.

Do you have one or two obviously large data structures than you can temporarily reduce, or large functions / libraries that you can temporarily stub out, to get it to build under Inspect, so you can see all the details on all the rest? It doesn't have to run, just compile and link....
 
Last edited:
Does it build normally, outside of the PlatformIO inspect function, either release or debug builds? It if does, what is the reported size from that build? There are several sections involved, any one of them being exceeded will prevent the linker, and knowing which section overflows is a clue to where your issue is.

Yeah, release build compiles and works - depending on which features of my project I enable, I can end up with anything from ~10K upwards free in the working build, eg my current configuration with a lot of things disabled gives me:-

Code:
Building .pio\build\pcb_go\firmware_pcb_go.hex
teensy_size: Memory Usage on Teensy 4.1:
teensy_size:   FLASH: code:331872, data:103584, headers:8956   free for files:7682052
teensy_size:    RAM1: variables:158304, code:312632, padding:15048   free for local variables:38304
teensy_size:    RAM2: variables:17568  free for malloc/new:506720

While an 'Inspect' build of the same configuration consistently comes out at a larger size and refuses to finish because of it:-

Code:
teensy_size: Memory Usage on Teensy 4.1:
teensy_size: FLASH: code:459256, data:94372, headers:8544 free for files:7564292
RAM1: variables:146048, code:423000, padding:2984 free for local variables:-47744
teensy_size: RAM2: variables:17568 free for malloc/new:506720
Error program exceeds memory space
*** [size] Error 4294967295
========================= [FAILED] Took 14.47 seconds =========================
Environment Status Duration
------------- -------- ------------
pcb_go FAILED 00:00:14.467
==================== 1 failed, 0 succeeded in 00:00:14.467 ===================="


Relevant Post on platformio forum about why debug build is required for 'Inspect':-
At the moment the “Project Inspector” only works with ELF files compiled in debug mode. The main reason is that we need additional information (like symbol locations) to prepare a comprehensive report. A map file is a potential workaround, but we support so many different platforms with different toolchain versions that makes parsing map files a pretty complicated task.

Avoiding the 'Inspect' tool and using some of these other map-based tools instead might be my best approach from here, although I have dabbled with those before and they're not as convenient as the 'Inspect' tool. (And maybe the platformio people will be able to advise better on whether I can do anything to ignore the 'exceeds memory space' error and continue inspection, thanks @PaulStoffregen )

It does seem quite strange to me that platformio will quite happily compile, link and upload a too-large firmware file that won't run on the hardware when using the 'Upload' task, but refuses to finish analysing a too-large firmware file in the 'Inspect' tool where it shouldn't even matter!

Perhaps these are things best solved by understanding how the platformio tasks are defined and seeing if I can solve this discrepancy there. I'm just quite hazy on how much of this is done by the platformio team and how much by pjrc and who to ask and where to find such information.

(As another example, 'Upload' task never displays the teensy_size output, and will blindly upload the firmware to the Teensy even if it is too large to run -- I have to use 'Build' task to see the teensy_size output to be told that it exceeds size.)


Do you have one or two obviously large data structures than you can temporarily reduce, or large functions / libraries that you can temporarily stub out, to get it to build under Inspect, so you can see all the details on all the rest? It doesn't have to run, just compile and link....

No obviously large data structures -- I've long since optimised all the obvious things away in order to make more space (by instantiating objects at setup and malloc'ing large arrays)!

My project has at least a dozen different build-time switches that turn on or off different functionality (eg enable/disable the 'custom behaviours' for all the different USB-MIDI devices that can be attached, different types of USB device support, disable the MIDI looper behaviour, etc...) -- picking and choosing which features/devices to support for a particular build is the only way that I can build something that'll run now as I've crammed so many things into it!

I use quite a lot of (sub-)classes and templated classes in this project too, and the 'Inspect' output was previously quite useful for understanding when such approaches were inflating binary size (eg by duplicating code for each template type) -- figuring out which coding styles end up generating the largest/most redundant code is where I'm having to focus now it seems, rather than looking for long-hanging data structures :)

(A previous attempt of mine to solve some low-hanging static data.. 2 years old.. this is a long journey I have been on with a project whose features are creeping like nobody's business!: https://forum.pjrc.com/index.php?th...cate-midi-tx-rx-buffers-non-statically.71610/)

Thanks for the thoughts and help, everybody, if nothing else it has been useful to articulate where I am with this!
 
Back
Top