Command Line Build

[DrM]

Is there a way to automatically create (or dump) a makefile to build a sketch from the command line, in linux?

And within the IDE, is there a way to set a macro definition, i.e. like #define in a source file or -Dmacroname in gcc?

[blackketter]

Check out https://platformio.org for command-line builds. You can specify gcc options in the platformio.ini file for a given project.

[DrM]

@blackketter I am sorry, that is not a good answer for me. I do not want to install more clutter, and I have no use or interest, none whatsoever, in a "a professional collaborative platform of embedded development". Uggh! I'm shaking at the thought of it.

Again, I just want a simple make command line, that I can add switches to for conditional builds. The context is that I am deploying the project on a few different flavors of board.

[brtaylor]

I'm a huge proponent of CMake. For my builds that have options, like which microcontroller to compile for, I use definitions so that I can choose between them on the command line. ie:

cmake .. -DMCU=MK66FX1M0

Which is a definition I'm using to select the MCU. CMake generates the appropriate Makefile and seems to do a good job caching builds, so compilation is quicker after the first one.

[WMXZ]

Again, I just want a simple make command line, that I can add switches to for conditional builds. The context is that I am deploying the project on a few different flavors of board.

So what is wrong with the makefile in cores?

[DrM]

@WMXZ Sounds great. Where is it in the tree? And, how do I use it?

Do I just copy it into the directory with the .ino file, or do I need to edit the makefile?

And how do I pass the switch for the conditional build? Is it just the usual CFLAGS=-Dsomething ?

And does it produce a hex file, or does it flash the teensy?

Thank you

[WMXZ]

@WMXZ Sounds great. Where is it in the tree? And, how do I use it?

Do I just copy it into the directory with the .ino file, or do I need to edit the makefile?

And how do I pass the switch for the conditional build? Is it just the usual CFLAGS=-Dsomething ?

And does it produce a hex file, or does it flash the teensy?

Thank you

Sorry, I thought you know how to read and use makefiles.
I don't know your system, I'm not using linux, but it is in the directory where the Teensy cores files are. (either Teensy3 or Teensy4)

[DrM]

Sorry, I thought you know how to read and use makefiles.
I don't know your system, I'm not using linux, but it is in the directory where the Teensy cores files are. (either Teensy3 or Teensy4)

I do indeed know how to read make files.

I still don't know where this one is to be found.

And sometimes makefiles are so complicated that it takes a long time to decipher what a particular makefile is doing and how to use it. And, I see from what is displayed on the ide that this a pretty complicated build

A little specific guidance and would be a big help and likely save a lot of time,

[WMXZ]

I do indeed know how to read make files.

I still don't know where this one is to be found.

And sometimes makefiles are so complicated that it takes a long time to decipher what a particular makefile is doing and how to use it. And, I see from what is displayed on the ide that this a pretty complicated build

A little specific guidance and would be a big help and likely save a lot of time,

Sure,
where you install TD you will find the following path (Windows language)

....\hardware\teensy\avr\cores\teensy4

where all the core filers of TD are (including the makefile) for T3.x use teensy3

[DrM]

Good, thank you, presumably you meant teensy3, in any case I found it.

It looks like it is setup for .c files and no obvious reference to the .ino files created by the ide.

What needs to be done to process .ino files?

Is it sufficient to simply add it to the objs (near the bottom of the makefile) analogous to the way the .c and .cpp files are added, or perhaps simply copy the .ino to a .c file?

[PaulStoffregen]

What needs to be done to process .ino files?

Usually these steps.

1: concatenate all .ino files together (if you have more than 1 .ino file) and rename to .cpp

2: add #include "Arduino.h" at the top

3: add function declarations as needed

This 3rd step usually involves some trial and error (or recompile). Arduino's preprocessing scans the .ino files for all the functions you've created and automatically puts function prototype declarations at the top of converted .cpp, so functions can appear in any order in the files. Normally C++ requires you to declare a function prototype if you use the function name in any part of the file before the actual function.

The exact details of how Arduino does this have changed over the years, so there isn't 1 simple and universal answer to this question.

The other piece of magic Arduino does is finding all the #include lines you have typed (or which are inside of libraries) and it tries to infer which libraries you meant to use. The result is -I include lines while compiling code, and building those libraries, and the final linker command which puts them into your compiled code.

For example if you include SD.h, Arduino's build system figures out you want the SD library, and inside the SD library it notices the SPI library is also needed. This too has changed over the years with different Arduino releases. It's quite a challenge and far beyond the scope of a makefile (but it is the domain of systems like autoconf) so you generally won't find makefiles which do this. Normally you have to edit the makefile for the libraries you want to build.

If you are hoping to craft a makefile which seamlessly "just works" with all .ino files, this automatic discovery of the required libraries isn't something makefiles are generally able to do. That's why tools like autoconf exist. Or maybe Cmake's more advanced features could do it? But if you also desire a small and simple makefile that's easy to read, I'm afraid seamlessly supporting all .ino files which include libraries with such a simple makefile is an impossible dream.

[DrM]

Hi Paul, I thought I replied earlier but perhaps I forgot to click post. So apologies if this is a repeat.

Is there a makefile that I could crib from? My includes are ADC.h, EEPROM.h and kinetis.h.

When I started down this path, I was hoping the IDE would be capable of generating a makefile.

[DrM]

Has anybody tried this:

https://github.com/sudar/Arduino-Makefile


It has a Teensy.mk

[DrM]

I found a possible answer, here

https://forum.arduino.cc/index.php?topic=713456.0

The following command seems to work, in so far as uploading or verifying,

# arduino  --pref compiler.cpp.extra_flags=-Dsomething   [ --verify | --upload ]  myfile.ino

The reply there says the --pref argument, compiler.cpp.extra_flags..., needs to be replaced by

teensy41.build.flags.defs=-D__IMXRT1062__ -DTEENSYDUINO=153 -Dmyflag

I am not sure what the specifics should be for the Teensy 3.2, and I have not checked whether the macro is being set.

Can anyone clarify this, or suggest the exact command line for the Teensy 3.2?

[luni]

Maybe I misunderstand something, but couldn't you simply call the arduino builder? Here a makefile I generated for you with VisualTeensy in "Ino-mode". The makefile is for Win10. Depending on the OS you are using you might need to adjust paths backslashes shell commands etc.

The important things is the FQBN. You find information how to assemble this online. Since this makefile uses the builder it will behave exactly as the IDE (which also uses the builder). I.e. it will compile *.inos, auto resolve the libraries etc...

#******************************************************************************
# Generated by VisualTeensy (https:\\github.com\luni64\VisualTeensy)
#
# Board              Teensy 3.2 \ 3.1
# USB Type           Serial
# CPU Speed          96 MHz (overclock)
# Optimize           Faster
# Keyboard Layout    US English
#
# 15.11.2020 15:19
#******************************************************************************
SHELL            := cmd.exe
export SHELL

SKETCH_NAME      := ttt_motor.ino

BOARD_ID         := TEENSY32
MCU              := mk20dx256

BUILDER_BASE     := "C:\Program Files\Arduino\arduino-1.8.12"
FLAGS            := -verbose=1 -warnings=all
FQBN             := teensy:avr:teensy31:usb=serial,speed=96,opt=o2std,keys=en-us
BUILD_PATH       := "C:\Users\lutz\source\Teensy\TeensySupport\ttt_motor\.vsteensy\build"
BUILD_CACHE      := "C:\Users\lutz\source\Teensy\TeensySupport\ttt_motor\.vsteensy\cache"
HARDWARE         := -hardware="C:\Program Files\Arduino\arduino-1.8.12/hardware"
TOOLS            := -tools="C:\Program Files\Arduino\arduino-1.8.12\tools-builder"

LIBRARIES        := -built-in-libraries="C:\Program Files\Arduino\arduino-1.8.12/libraries"
LIBRARIES        := -libraries="C:\Users\lutz\Documents\Arduino\libraries"
UPL_PJRC_B       := C:\PROGRA~1\Arduino\ARDUIN~1.12\hardware\tools
UPL_TYCMD_B      := C:\toolchain\TyTools
UPL_CLICMD_B     := 

TARGET_HEX       := $(BUILD_PATH)/$(SKETCH_NAME).hex

#******************************************************************************
# BINARIES
#******************************************************************************
BUILDER     := $(BUILDER_BASE)/arduino-builder
UPL_PJRC    := "$(UPL_PJRC_B)/teensy_post_compile" -test -file=$(SKETCH_NAME) -path=$(BUILD_PATH) -tools="$(UPL_PJRC_B)" -board=$(BOARD_ID) -reboot
UPL_TYCMD   := $(UPL_TYCMD_B)/tyCommanderC upload $(TARGET_HEX) --autostart --wait --multi
UPL_CLICMD  := $(UPL_CLICMD_B)/teensy_loader_cli -mmcu=$(MCU) -v $(TARGET_HEX) 

.PHONY:  all upload uploadTy uploadCLI clean 

all:  
	$(BUILDER) $(FLAGS) -build-path=$(BUILD_PATH) -build-cache=$(BUILD_CACHE) -fqbn=$(FQBN) $(HARDWARE) $(TOOLS) $(LIBRARIES) $(SKETCH_NAME)

upload: all
	$(UPL_PJRC)

uploadTy: all
	@$(UPL_TYCMD)

uploadCLI: all
	@$(UPL_CLICMD)

clean: 	
	@echo Cleaning user binaries...
	@if exist .vsteensy\build rd /s/q .vsteensy\build
	@md .vsteensy\build

	@if exist .vsteensy\cache rd /s/q .vsteensy\cache"
	@md .vsteensy\cache
	@echo done

[KurtE]

Can anyone clarify this, or suggest the exact command line for the Teensy 3.2?

Probably the best way to answer some questions like this, is to simply run the Arduino build and see what command lines that are used.

That is go in to preferences and turn on verbose compile and verbose upload and see what all of the commands are that are generated.

You can also deduce what some of these settings would be by looking at the file boards.txt that Teensyduino installs for your specific board.

[luni]

I just saw that you want to pass additional defines to the compiler. This can also be done easily but you need to tweak platform.txt a little bit. Scroll down to the ##compile c++ files section and add {build.extra_flags} to the recipe (unfortunately the stock platform.txt doesn't define this).

## Compile c++ files
recipe.cpp.o.pattern="{compiler.path}{build.toolch ain}{build.command.g++}" -c {build.extra_flags} {build.flags.optimize} {build.flags.common} {build.flags.dep} {build.flags.cpp} {build.flags.cpu} {build.flags.defs} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DF_CPU={build.fcpu} -D{build.usbtype} -DLAYOUT_{build.keylayout} "-I{build.path}/pch" {includes} "{source_file}" -o "{object_file}"

With that, you can pass additional flags to the compiler by adding the -prefs option to the builder command line. Here a quick hack in the makefile from above (of course that should be done with some variables...)

#******************************************************************************
# BINARIES
#******************************************************************************
BUILDER     := $(BUILDER_BASE)/arduino-builder
UPL_PJRC    := "$(UPL_PJRC_B)/teensy_post_compile" -test -file=$(SKETCH_NAME) -path=$(BUILD_PATH) -tools="$(UPL_PJRC_B)" -board=$(BOARD_ID) -reboot
UPL_TYCMD   := $(UPL_TYCMD_B)/tyCommanderC upload $(TARGET_HEX) --autostart --wait --multi
UPL_CLICMD  := $(UPL_CLICMD_B)/teensy_loader_cli -mmcu=$(MCU) -v $(TARGET_HEX)

.PHONY:  all upload uploadTy uploadCLI clean

all:
	$(BUILDER) $(FLAGS) -prefs=build.extra_flags=-DMY_DEFINE=42 -build-path=$(BUILD_PATH) -build-cache=$(BUILD_CACHE) -fqbn=$(FQBN) $(HARDWARE) $(TOOLS) $(LIBRARIES) $(SKETCH_NAME)

And here a part of the compile output which shows that MY_DEFINE is actually passed to the compiler:

sy\\TeensySupport\\ttt_motor\\.vsteensy\\build\\sketch\\ttt_motor.ino.cpp" -o "C:\\Users\\lutz\\source\\Teensy\\TeensySupport\\ttt_motor\\.vsteensy\\build\\sketch\\ttt_motor.ino.cpp.o"
"C:\\Program Files\\Arduino\\arduino-1.8.12\\hardware\\teensy/../tools/arm/bin/arm-none-eabi-g++" -c -DMY_DEFINE=42 -O2 -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD -fno-exceptions -fpermissive -felide-constructors -std=gnu++14 -Wno-error=narrowing -fno-rtti -mthumb -mcpu=cortex-m4 -fsingle-precision-constant -D__MK20DX256__ -DTEENSYDUINO=154 -DARDUINO=10600 -DARDUINO_TEENSY32 -DF_CPU=96000000 -DUSB_SERIAL -DLAYOUT_US_ENGLISH "-IC:\\Users\\lutz\\source\\Teensy\\TeensySupport\\ttt_motor\\.vsteensy\\build/pch" "-IC:\\Program Files\\Arduino\\arduino-1.8.12\\hardware\\teensy\\avr\\cores\\teensy3"

Hope that helps...

[DrM]

@KurtE that sounds like a simple approach, I was thinking along those lines, but did not find the switch for verbose in the gui.

@luni that seems very interesting. Are build_path and build_cache specific to your project? I just have one ino file, with includes for adc and eeprom. Is there not a simple one liner for that using the arduino command?

[luni]

Are build_path and build_cache specific to your project? I just have one ino file, with includes for adc and eeprom. Is there not a simple one liner for that using the arduino command?

Honestly, I don't understand the question. This

$(BUILDER) $(FLAGS) -prefs=build.extra_flags=-DMY_DEFINE=42 -build-path=$(BUILD_PATH) -build-cache=$(BUILD_CACHE) -fqbn=$(FQBN) $(HARDWARE) $(TOOLS) $(LIBRARIES) $(SKETCH_NAME)

obviously IS a simple one liner? I'm afraid it won't get any simpler than calling the arduino builder with a bunch of parameters. It obviously needs some information about where to find libraries, tools, where to place the binaries and for which board and board settings it should compile. Here https://github.com/arduino/arduino-builder the doc about the available parameters.

Of course you need to adjust the paths and parameters to your environment. The makefile I posted above obviously uses my setup but since you mentioned that you are familiar with makefiles I assumed that it shouldn't be difficult to translate to your environment. You can also call the builder manually from the shell or from a bash script if you prefer. Just give it the correct parameters and it will compile your inos. You can compile for your different boards/options by simply calling the builder with different FQBNs. I suggest to also use a different build_path per board to have the hex files separated.

[DrM]

@luni

You have these lines in your make file. I am wondering how they would translate to my project, or if they are needed.

BUILD_PATH       := "C:\Users\lutz\source\Teensy\TeensySupport\ttt_motor\.vsteensy\build"
BUILD_CACHE      := "C:\Users\lutz\source\Teensy\TeensySupport\ttt_motor\.vsteensy\cache"

[luni]

These define where the builder stores the binaries and the cache file. You can choose any path you like for them. Did you read the documentation I linked in the last post?

[DrM]

These define where the builder stores the binaries and the cache file. You can choose any path you like for them. Did you read the documentation I linked in the last post?

What? Read the document?

Actually, i didn't notice the link. I'll take a look at it. Thank you

[manicksan]

as you stated in post Yesterday, 02:53 PM #14

why not use this mode of Arduino IDE?

they have some undocumented arguments as well
look at:

https://github.com/arduino/Arduino/b...ineParser.java

there is for example a --buildpath argument that you can use to select where the output files should go,
using this makes it easier to debug some stuff
to use the buildpath the directory must exist before running arduino --buildpath [path]

and the build flags should be:

--pref build.flags.cpp=

or

--pref teensy41.build.flags.cpp=

if you want to specify the target

if you look at hardware\teensy\avr\boards.txt in arduino install folder
you can see the build flags there


in windows you should use arduino_debug instead of arduino if you want to see any output
also using the --verbose argument shows compile outputs

example: on windows 10 (verified)

arduino_debug --verbose --board teensy:avr:teensy40 --pref "build.flags.cpp=-D USB_MIDI_AUDIO_SERIAL -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing" --verify sketch_oct10a.ino

example: on ubuntu (verified)

arduino --verbose --board teensy:avr:teensy40 --pref 'build.flags.cpp=-D USB_MIDI_AUDIO_SERIAL -std=gnu++14 -fno-exceptions -fpermissive -fno-rtti -fno-threadsafe-statics -felide-constructors -Wno-error=narrowing' --verify sketch_oct10a.ino

[luni]

if you look at hardware\teensy\avr\boards.txt in arduino install folder
you can see the build flags there

Isn't it a bit tedious and error prone to read out all those flags from boards.txt manually? Also some of those flags have dependencies which makes it even more complicated. The builder extracts them automatically by providing the FQBN. You just need to add your additional flags. Maybe your Arduino command also supports FQBNs instead of the raw defines?

[manicksan]

I just wanted do describe where I found the "build.flags.cpp" flag from

I did some testing and found out that you should use it like this:
on windows

arduino_debug --verbose --board teensy:avr:teensy40:usb=serialmidiaudio --verify sketch_oct10a.ino

on ubuntu/linux also maybe macos

arduino_debug --verbose --board teensy:avr:teensy40:usb=serialmidiaudio --verify sketch_oct10a.ino

on macos catalina (here it needs the buildpath otherwise teensyloader was not able to read the .hex file)
note that you need to create the buildTemp folder beforehand,
also the GUI splash screen seems to load with all initiations included which means it's taking longer time

'/Applications/Teensyduino.app/Contents/MacOS/Arduino' --verbose --board teensy:avr:teensy40:usb=serialmidiaudio --buildpath ~/Documents/Arduino/buildTemp --verify ~/Documents/Arduino/sketch_oct18b/sketch_oct18b.ino

so the build flag is not needed (only if you want to use additional global include files/global definitions)

I was missing the usb=mode
where mode can be: (yet again taken from the boards.txt file)

mode          description (as in the menu)
---------------------------
serial
serial2       Dual Serial
serial3       Triple Serial
keyboard
touch         Keyboard + Touch Screen
hidtouch      Keyboard + Mouse + Touch Screen
hid           Keyboard + Mouse + Joystick
serialhid     Serial + Keyboard + Mouse + Joystick
midi
midi4
midi16
serialmidi
serialmidi4
serialmidi16
audio
serialmidiaudio
serialmidi16audio
mtp                 MTP Disk (Experimental)
rawhid
flightsim
flightsimjoystick
disable