-flto with T4 (Teensyduino 1.58)

pr8x · Apr 23, 2023

So I am working on bigger project with quite a few external dependencies. It seems that despite -Os some of these still have quite the code bloat and my T4 is running out of memory.
These seem to be the biggest offenders at the moment:

Forking each of these libraries and removing things I don't need/use (like removing the Opus from the Codecs library) could be an option, but I wonder if the compiler could be smart enough to do that. I think with LTO (at least in theory) it should have all the information to remove unsued objects. I tried adding it to my build flags but it doesn't seem to work:

C:\Users\X\.platformio\packages\framework-arduinoteensy@src-0883a3d713cdb3613f9056dcc751d771\cores\teensy4\EventResponder.cpp:45:9: error: '_serialEvent_default' causes a section type conflict with 'usb_config_descriptor_12'
45 | uint8_t _serialEvent_default __attribute__((weak)) PROGMEM = 0 ;
| ^
C:\Users\X\.platformio\packages\framework-arduinoteensy@src-0883a3d713cdb3613f9056dcc751d771\cores\teensy4\usb_desc.c:1669:23: note: 'usb_config_descriptor_12' was declared here
1669 | PROGMEM const uint8_t usb_config_descriptor_12[CONFIG_DESC_SIZE] = {
| ^
lto-wrapper.exe: fatal error: arm-none-eabi-g++ returned 1 exit status

Saw similar errors mentioned here and the fix was to make the static const. That cannnot be used with __attribute__((weak)) though...
Did anybody manage to make LTO work with latest Teensyduino 1.58 toolchain? And, more generally, any tips besides using PROGMEM (can't use that in external libraries everywhere) to reduce size of my firmware? Could priobably experiment with --gc-sections as well.

jmarsh · Apr 23, 2023

You shouldn't need LTO to prune unreferenced code and data. Passing -ffunction-sections and -fdata-sections to the compiler and --gc-sections to the linker should be enough but they have to be definitively unreferenced; that may not be the case if the audio library tries to automatically figure out which codec is required for a stream, for example.

Is there no global config header (or compile time defines) that you can use to exclude features when building the libraries?

pr8x · Apr 23, 2023

Tried compiling with -fdata-sections -ffunction-sections -Wl,--gc-sections. That doesn't seem to make much of a difference though.

luni · Apr 23, 2023

They are set by default anyway.

luni · Apr 23, 2023

It seems that despite -Os some of these still have quite the code bloat and my T4 is running out of memory.

Looks like you are using PIO? Did you also set "--specs=nano.specs"? Newlib nano is used by teensyduino if you choose smallest code.

pr8x · Apr 23, 2023

@luni Oh, I had no idea about this. This fixed the issue. Went from -2816 to 96000 for local variables. Awesome! What does it actually do? This should be the new default for PlatformIO.

luni · Apr 23, 2023

There are two versions of the standard library for this toolchain: newlib and newlib-nano. specs=nano.specs switches to the nano version of the libray. There probably is a reason why teensyduino only uses newlib-nano for -Os and the "big" newlib otherwise (Paul would know). I never found a disadvantage of newlib-nano. Here some information: http://support.raisonance.com/content/what-difference-between-newlib-and-newlib-nano.

BTW: If you need floating point printf you can activate this by "specs=nano.specs -u _printf_float" (same for scanf)

PaulStoffregen · Apr 23, 2023

pr8x said:
This should be the new default for PlatformIO.

What "should" be done is of course a matter of opinion, but my opinion is PlatformIO really ought to stay much closer to the set of options we offer in the Arduino IDE's Tools menu. Those are the options which are well tested.

luni · Apr 23, 2023

but my opinion is PlatformIO really ought to stay much closer to the set of options we offer in the Arduino IDE's Tools menu

The problem is that PIO doesn't have such a menu. You can easily set compiler options but you need to know that e.g. -Os goes with -specs=nano-spec in teensyduino. Something like a translation of the tools menu to compiler options might be worth to add to the PIO documentation.

pr8x · Apr 23, 2023

Yeah, PIO doesn't really provide any predefined optimization profiles for Teensy AFAICT. You can only set raw compiler flags. Are there any other flags that Teensyduino uses that I have to manually set in PIO?

PaulStoffregen · Apr 23, 2023

luni said:
Something like a translation of the tools menu to compiler options might be worth to add to the PIO documentation.

Maybe PlatformIO could add commented examples in their default config file which exactly match the Arduino IDE Tools menu options?

Perhaps some actual comments could strongly advise using those well tested settings and if using customized settings, to first check whether any discovered problems are reproducible with the official settings.

luni · Apr 23, 2023

@pr8x: Here (windows)

Code:

<user>\.platformio\packages\framework-arduinoteensy

you find board.txt. Once you understand the unusual file format you can easily find the required info

luni · Apr 23, 2023

I stand corrected. As usual: "If nothing helps, read the fucking manual"

https://docs.platformio.org/en/latest/platforms/teensy.html#platform-teensy

One doesn't need to directly set the compiler flags, there are nice macros doing this. E.g. this configuration:

Code:

[env:teensy41]
platform = teensy
board = teensy41
build_flags = [COLOR="#FF0000"]-D TEENSY_OPT_SMALLEST_CODE[/COLOR]
framework = arduino

is setting up everything correctly. Same works for the USB options.

Also see: https://forum.pjrc.com/threads/72623-platformio-ini-for-Teensy-4-1

PaulStoffregen · Apr 23, 2023

Does this file allow comments?

Can we get the default confirm PlatformIO distributes to mention this and maybe even a link to the docs in comments? It's one thing to say RTFM here on the forum after someone has already struggled with guessing settings. The place where this info (basically RTFM) needs to be said is in the comments in the copy they see when first trying to edit the settings.

luni · Apr 24, 2023

PaulStoffregen said:
Does this file allow comments?

Can we get the default confirm PlatformIO distributes to mention this and maybe even a link to the docs in comments? It's one thing to say RTFM here on the forum after someone has already struggled with guessing settings. The place where this info (basically RTFM) needs to be said is in the comments in the copy they see when first trying to edit the settings.

This is what is autogenerated for a new Teensy (4.1) project.

Code:

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[env:teensy41]
platform = teensy
board = teensy41
framework = arduino

The documentation link is generic and does not (directly) lead to board/platform specific pages. I doubt that it is possible to add board specific information to the auto generated config file. But I'm not a frequent PIO user. Others will know more about this.

The teensy specific documentation is on GitHub
- https://github.com/platformio/platformio-docs/blob/develop/platforms/teensy.rst
- https://github.com/platformio/platformio-docs/blob/develop/platforms/teensy_extra.rst

Don't know if they accept pull requests in case something needs to be improved

geekEE · May 12, 2023

I also found it difficult to find the build options for Teensy 4.0 in PlatformIO. I found this page (https://docs.platformio.org/en/latest/platforms/teensy.html) lists a number of optimization options, but it says "valid only for Teensy LC, Teensy 3.0-3.6". However, when I look at the code in home/.platformio/platforms/teensy/builder/frameworks/arduino.py, I believe many of them should work for Teensy 4.x.

Edit: I just noticed that Luni had a link that led to the same information, but on Github. I guess I should read more carefully.

sbfreddie · May 13, 2023

Update USB options

All:
Here is a General Template for a PlatformIO.ini file:

Code:

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[env:development]
platform = https://github.com/platformio/platform-teensy.git

board = teensy31
;board = teensy41

framework = arduino

build_flags =
    -Wl,--print-memory-usage
    -g
    -D USB_MTPDISK_SERIAL

check_tool = cppcheck

check_flags = --enable=all
;
extra_scripts = 
	pre:version_increment_pre.py
	post:version_increment_post.py
;
;build_type = debug
;  
;lib_deps =
	# RECOMMENDED
	# Accept new functionality in a backwards compatible manner and patches
	#adafruit/Adafruit BusIO @ ^1.9.0

	# Accept only backwards compatible bug fixes
	# (any version with the same major and minor versions, and an equal or greater patch version)
 	 #adafruit/Adafruit BusIO @ ~1.9.0

	# The exact version
	#adafruit/Adafruit BusIO @ 1.9.0
;
;	https://github.com/ftrias/TeensyDebug/archive/refs/heads/master.zip 
;	thijse/EEPROMEx@0.0.0-alpha+sha.09d7586108
;	adafruit/Adafruit SleepyDog Library@^1.6.0
;	sparkfun/SparkFun BME280@^2.0.9
;	milesburton/DallasTemperature@^3.9.1
;	jchristensen/Timezone@^1.2.4    
;    
;build_flags =
    #	USB Features
	#	If you want to use Teensy USB Features, you need to add special macro/define using build_flags:
;	-D USB_SERIAL
;	-D USB_DUAL_SERIAL
;	-D USB_TRIPLE_SERIAL
;	-D USB_KEYBOARDONLY
;	-D USB_TOUCHSCREEN
;	-D USB_HID_TOUCHSCREEN
;	-D USB_HID
;	-D USB_SERIAL_HID
;	-D USB_MIDI
;	-D USB_MIDI4
;	-D USB_MIDI16
;	-D USB_MIDI_SERIAL
;	-D USB_MIDI4_SERIAL
;	-D USB_MIDI16_SERIAL
;	-D USB_AUDIO
;	-D USB_MIDI_AUDIO_SERIAL
;	-D USB_MIDI16_AUDIO_SERIAL
;	-D USB_MTPDISK
;	-D USB_RAWHID
;	-D USB_FLIGHTSIM
;	-D USB_FLIGHTSIM_JOYSTICK
;	-D USB_EVERYTHING
;	-D USB_DISABLED
;A default macro is set to -D USB_SERIAL if none is specified.
    
;debug_port = /dev/cu.usbmodem61684903  ;Or something similar
;debug_tool = custom
;debug_load_mode = manual
;debug_server = 
;debug_init_cmds =
  ;target extended-remote $DEBUG_PORT
  ;$INIT_BREAK
  ;define pio_reset_run_target
  ;interrupt
  ;tbreak loop
  ;continue
  ;end
  ;define pio_restart_target
  ;echo Restart is undefined for now.
  ;end
;
;debug_init_break =


; change MCU frequency
;board_build.f_cpu = 600000000L	;600MHz for Teensy 4.1
;board_build.f_cpu = 450000000	;450MHz for Teensy 4.1
;board_build.f_cpu = 180000000L	;180MHz for Teensy 3.6
;board_build.f_cpu = 120000000L	;180MHz for Teensy 3.5
;board_build.f_cpu = 72000000L	;72MHz to 96MHz for Teensy 3.2

;	Optimization
;You can control firmware optimization via special macro/define using build_flags in “platformio.ini” (Project Configuration File):
;build_flags =     
;   -D IMPROVE_HIGH_FREQUENCY_ACCURACY
;   -D IMPROVE_EXPONENTIAL_ACCURACY
;   -D TEENSY_OPT_FASTER, default
;   -D TEENSY_OPT_FASTER_LTO
;   -D TEENSY_OPT_FAST
;   -D TEENSY_OPT_FAST_LTO
;   -D TEENSY_OPT_FASTEST
;   -D TEENSY_OPT_FASTEST_LTO
;   -D TEENSY_OPT_FASTEST_PURE_CODE, valid only for Teensy 3.5-3.6
;   -D TEENSY_OPT_FASTEST_PURE_CODE_LTO, valid only for Teensy 3.5-3.6
;   -D TEENSY_OPT_DEBUG
;   -D TEENSY_OPT_DEBUG_LTO
;   -D TEENSY_OPT_SMALLEST_CODE
;   -D TEENSY_OPT_SMALLEST_CODE_LTO
;
;The only one macro can be used in per one build environment. Also, you can see verbose build using -v, --verbose option for pio run command.
;
;
;Uploading using CLI
;
; By default the Teensy platform uses the graphical version of the Teensy upload tools.
; For cases where the GUI tools cannot be used (e.g. headless OS)
; the native Teensy Loader tool is also available in a command line version
; which can be enabled directly in your project configuration, for example:
;
;upload_protocol = teensy-cli

;	-DIMPROVE_HIGH_FREQUENCY_ACCURACY
;	-DIMPROVE_EXPONENTIAL_ACCURACY

;lib_extra_dirs = ~/Documents/Arduino/libraries

; Do not include folders /lib/console and /misc
;[env:no_console]
;src_filter = ${env.src_filter} -<../lib/ina226/> -<../misc/>

;It could be very useful when working on new library versions, comparing them against the old, currently installed within the framework ones. 
;Simply add the new version in the /lib folder and create a conditional build, ie:
;[platformio]
;default_builds = stock_lib
;[env]
; define board, framework, platform etc here
;[env:stock_build]
;src_filter = ${env.src_filter} -<../lib/new_lib_test_version/>
;If the default_libs is set to stock_lib, the local library folder will not be included and the compiler will grab+link the global version.
;

Regards,
Ed

-flto with T4 (Teensyduino 1.58)

pr8x

Member

jmarsh

Well-known member

pr8x

Member

luni

Well-known member

luni

Well-known member

pr8x

Member

luni

Well-known member

PaulStoffregen

Well-known member

luni

Well-known member

pr8x

Member

PaulStoffregen

Well-known member

luni

Well-known member

luni

Well-known member

PaulStoffregen

Well-known member

luni

Well-known member

geekEE

Active member

sbfreddie

Well-known member