[?] Cross compiling bypassing completely Arduino IDE

Status
Not open for further replies.
K

killkrt

Member
Hi,

I am making some experiments with Teensy, especially developing audio applets using the great Ornament & Crime module.

I found the Arduino IDE too basic to be used for serious development, and I switched to VSCode + Platformio, but anyway every time I have to open Arduino IDE to compile.

I would like to know if there is any tutorials on how to compile using directly the cross compiler, I am going to start a new project and I would prefer to write my own my Makefile and keep things under control (and have unit tests running before downloading).
It would be already nice to have a dump of the build commands executed by Teensyduino, to guess how to use it.

Thank you.
 
An example makefile comes with the Teensyduino installation. And you might do a few compiles with verbose in the Arduino IDE, using different optimizations to see the different command line parameters for the compiler and linker, and use these in your own build system.
 
Thank you the very quick answer.

I am wondering if I could just reverse engineering the verbose output of Arduino IDE or maybe Teensyduino makes some other operations on top of it (I mean something not logged in the verbose dump).
 
I wouldn‘t have mentioned the verbose output if I had not considered it being useful. I‘m sure that you saw already the definitions in Teensyduino‘s boards.txt file, so you‘d know how to put everything together.
 
You can have a look at a Makefile I use here:
https://github.com/bolderflight/RAPTRS/blob/master/software/Makefile

It's cross building more than just the Teensy software, but if you follow the 'FMU' and 'NODE' variables you'll see all my steps for compiling for Teensy 3.6. It's based on the Makefile packaged with Teensyduino; although, with fewer options (I only build for Teensy 3.6 and a CPU of 240 MHz). I'm also using the newest gcc and compiling with c++17. If you google search for 'Teensy Makefile', you'll also find similar examples. It's really quite straightforward if you understand GNU Make.
 
Using @luni's linked : VisualTeensy might be a good place to start to see a makefile. His effort isn't two months old but works quite well. The VisualCode environment is cross platform and Luni's work to capture and extend the elements that go into Arduino and Teensy with a makefile are impressive.
 
Hi,

Finally I've received my Teensy 3.6 and started to work with it.
I've created a basic Makefile (following the VisualTeensy Makefile, thanks @defragster) and adapted for Linux, but I have two problems:

  • Serial output seems faulty: I can only see few lines (using minicom or screen or cat) and then it seems the connection hangs, while if I compile the same code with Arduino IDE it works perfectly.
  • Some C++ STD parts seems to be not defined: for example it seems that I cannot use "this_thread" since it is not defined in threads.h. I was thinking all STD was ported for the Teensy.

Please find in attachment my Makefile and the little source code.

Thank you.
 

Attachments

  • TeesnyTest.zip
    3 KB · Views: 346
killkrt said:
Hi,
… (following the VisualTeensy Makefile, thanks @defragster) ...

  • Serial output seems faulty: I can only see few lines (using minicom or screen or cat) and then it seems the connection hangs, while if I compile the same code with Arduino IDE it works perfectly.
  • Some C++ STD parts seems to be not defined: for example it seems that I cannot use "this_thread" since it is not defined in threads.h. I was thinking all STD was ported for the Teensy.
...Thank you.
Click to expand...

Thanks go to @luni!

If IDE works then assume the 'rules' were applied to allows the device access?

You could look up TyCommander - seems you need to build that for yourself on Linux? - but it offers a great SerMon and loader as can be seen in VisualTeensy.

If the STD _thread works in IDE it was ported … if not then it was skipped for some reason.
 
Last edited:
Also, a too modern toolchain may contain 'incompatibilities' with Teensy core files.
( I had that years ago, when I tried to use a off-the web toolchain)
Now I stick with the one Teensyduino provides
 
Was going to mention TeensyThreads - but reading the OP - I didn't find any notes suggesting how it was to help or be used ...
 
Cool! TyCommander is working perfectly! Anyway I cannot understand what I was missing using minicom, anyway my first issue is solved!
 
killkrt said:
Thank you for the reply @brtaylor.
Where I can find a documentation about STD port coverage?

Thank you!
Click to expand...

Honestly, I had to poke around the ARM developers forum for that info when I ran into compiling a package that required std::threads, but I haven't found a definitive list. So far, in my experience, the rest of the STL works fine.
 
killkrt said:
Cool! TyCommander is working perfectly! Anyway I cannot understand what I was missing using minicom, anyway my first issue is solved!
Click to expand...

Great to hear! Thanks to @koromix for his great work there! I use precompiled on Windows - glad to hear you are seeing it well on your Linux.
 
@killkrt:
I had a quick look at your makefile and code and have the following remarks:

  • In your main.cpp you define the function
    Code: 
    int main()
{
  setup();
  while( true )
  {
    loop();
  }
  return 0;
}
    This is not necessary (and it wont be used) since this function is already defined in the core library. Just use setup() and loop() and you are fine. No need for redefining main.

  • There is a bug in my makefile which you used as a base. You need to add:
    Code: 
    CORE_S_FILES    := $(call rwildcard,$(CORE_SRC)/,*.S)
    to the "Core library sources" before the defintion of CORE_OBJ (i.e. before line 74) otherwise the makefile doesn't compile the memset.S und memcpy.S files. (If you wonder why it works nevertheless: gcc will use default implementations for those http://gcc.gnu.org/onlinedocs/gcc/Link-Options.html)

    If you are using TyCommander add --multi to the TyCMD options in line 54. This enables upload of the firmware to more than one teensy in parallel.
 
@luni

Thank you for the hints and for your great work with VisualTeensy, it helped me a lot in understanding how to write a working Makefile!
 
I too became quickly frustrated with the Arduino IDE (some years ago now). It's great for beginners but not for someone who has used a 'grown up' IDE...

I tried my own makefile which worked well but was a pain to edit (I don't really know make very well).

I tried the Eclipse Arduino plugin. It worked well but Eclipse is complex and has a bazillion options that are not applicable.

I tried Visual Micro. Very nice for me as I use Visual Studio for other things. The trouble is, they want you to pay for it!

Now I find VisualTeesny. Nice and lightweight. The source is available. With a bit of hacking I can get the makefile to work with VisualCode under Linux (I use both Windows and Linux). Excellent - many thanks @luni!
 
Hi Hughby,

I assume you have already found it. With Arduino you can also simply go into preferences, and set the option "Use External Editor", which will then show all of your Arduino windows in read only mode, and then edit in your favorite editor and each time you save any updates they show up in the Arduino IDE, where you can then issue the build or upload commands... Not as eloquent as using the Visual Studio stuff, but gets the job done...

I have typically stuck with this approach as if I share code and/or use shared code, having it build using the same assumptions and tools as Arduino helps to keep things consistent. But I typically have used Sublime Text to do most of my editing.
 
Glad that VisualTeensy is useful for you.

With a bit of hacking I can get the makefile to work with VisualCode under Linux
Click to expand...
Would it be possible to send me that tweaked Linux makefile? I could then add a "generate Linux compatible makefile" option if that would be usesful.
 
Certainly. Let me review what I did and I'll send you a project with with some comments on what I did. Much was obvious (paths etc.) but there was at least one change that took a bit of thinking about and more learning about make than I really wanted!
 
luni said:
Glad that VisualTeensy is useful for you.

Would it be possible to send me that tweaked Linux makefile? I could then add a "generate Linux compatible makefile" option if that would be usesful.
Click to expand...


Hmm... I produced a lovely .rtf file with changes highlighted - any idea how I can attach it here or send it to you via this forum @luni?

In the meantime here is the linux makefile

Code: 
#******************************************************************************
# Generated by VisualTeensy (https://github.com/luni64/VisualTeensy)
#
# Board              Teensy 3.2 / 3.1
# USB Type           Serial
# CPU Speed          96 MHz (overclock)
# Optimize           Faster
# Keyboard Layout    US English
#
# 06/11/2018 13:30
#******************************************************************************
#SHELL            := cmd.exe
#export SHELL

TARGET_NAME      := TeensyViewDemo
BOARD_ID         := TEENSY31

LIBS_SHARED_BASE := /home/hugh/arduino-1.8.7/libraries
LIBS_SHARED      := 

LIBS_LOCAL_BASE  := lib
LIBS_LOCAL       := SPI 

CORE_BASE        := /home/hugh/arduino-1.8.7/hardware/teensy/avr/cores/teensy3
GCC_BASE         := /home/hugh/arduino-1.8.7/hardware/tools/arm
UPL_PJRC_B       := /home/hugh/arduino-1.8.7/hardware/tools
UPL_TYCMD_B      := /home/hugh/Arduino
UPL_CLICMD_B     := 

MCU   := mk20dx256

FLAGS_CPU   := -mthumb -mcpu=cortex-m4 -fsingle-precision-constant
FLAGS_OPT   := -O2
FLAGS_COM   := -g -Wall -ffunction-sections -fdata-sections -nostdlib -MMD
FLAGS_LSP   := 

FLAGS_CPP   := -fno-exceptions -felide-constructors -std=gnu++14 -Wno-error=narrowing -fno-rtti
FLAGS_C     := 
FLAGS_S     := -x assembler-with-cpp
FLAGS_LD    := -Wl,--gc-sections,--relax,--defsym=__rtc_localtime=$(shell date +%s)

LIBS        := -larm_cortexM4l_math -lm
LD_SCRIPT   := mk20dx256.ld

DEFINES     := -D__MK20DX256__ -DTEENSYDUINO=144 -DARDUINO=10807
DEFINES     += -DF_CPU=96000000 -DUSB_SERIAL -DLAYOUT_US_ENGLISH

CPP_FLAGS   := $(FLAGS_CPU) $(FLAGS_OPT) $(FLAGS_COM) $(DEFINES) $(FLAGS_CPP)
C_FLAGS     := $(FLAGS_CPU) $(FLAGS_OPT) $(FLAGS_COM) $(DEFINES) $(FLAGS_C)
S_FLAGS     := $(FLAGS_CPU) $(FLAGS_OPT) $(FLAGS_COM) $(DEFINES) $(FLAGS_S)
LD_FLAGS    := $(FLAGS_CPU) $(FLAGS_OPT) $(FLAGS_LSP) $(FLAGS_LD)
AR_FLAGS    := rcs

USR_SRC     := src
LIB_SRC     := lib
CORE_SRC    := $(CORE_BASE)

BIN         := bin
USR_BIN     := $(BIN)/src
CORE_BIN    := $(BIN)/core
LIB_BIN     := $(BIN)/lib
CORE_LIB    := $(BIN)/core.a
TARGET_HEX  := $(BIN)/$(TARGET_NAME).hex
TARGET_ELF  := $(BIN)/$(TARGET_NAME).elf
TARGET_LST  := $(BIN)/$(TARGET_NAME).lst 

#******************************************************************************
# BINARIES
#******************************************************************************
CC          := $(GCC_BASE)/bin/arm-none-eabi-gcc
CXX         := $(GCC_BASE)/bin/arm-none-eabi-g++
AR          := $(GCC_BASE)/bin/arm-none-eabi-gcc-ar
OBJCOPY     := $(GCC_BASE)/bin/arm-none-eabi-objcopy
SIZE        := $(GCC_BASE)/bin/arm-none-eabi-size
OBJDUMP     := $(GCC_BASE)/bin/arm-none-eabi-objdump
UPL_PJRC    := "$(UPL_PJRC_B)/teensy_post_compile" -test -file=$(TARGET_NAME) -path=$(BIN) -tools="$(UPL_PJRC_B)" -board=$(BOARD_ID) -reboot
UPL_TYCMD   := $(UPL_TYCMD_B)/tycommander upload $(TARGET_HEX) --autostart --wait --multi
UPL_CLICMD  := $(UPL_CLICMD_B)/teensy_loader_cli -mmcu=$(MCU) -v $(TARGET_HEX)

#******************************************************************************
# Source and Include Files
#******************************************************************************
# Recursively create list of source and object files in USR_SRC and CORE_SRC 
# and corresponding subdirectories. 
# The function rwildcard is taken from http://stackoverflow.com/a/12959694)

rwildcard =$(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))

#User Sources -----------------------------------------------------------------
USR_C_FILES    := $(call rwildcard,$(USR_SRC)/,*.c)
USR_CPP_FILES  := $(call rwildcard,$(USR_SRC)/,*.cpp)
USR_S_FILES    := $(call rwildcard,$(USR_SRC)/,*.S)
USR_OBJ        := $(USR_S_FILES:$(USR_SRC)/%.S=$(USR_BIN)/%.o) $(USR_C_FILES:$(USR_SRC)/%.c=$(USR_BIN)/%.o) $(USR_CPP_FILES:$(USR_SRC)/%.cpp=$(USR_BIN)/%.o) 

# Core library sources --------------------------------------------------------
CORE_CPP_FILES := $(call rwildcard,$(CORE_SRC)/,*.cpp)
CORE_C_FILES   := $(call rwildcard,$(CORE_SRC)/,*.c)
CORE_OBJ       := $(CORE_S_FILES:$(CORE_SRC)/%.S=$(CORE_BIN)/%.o) $(CORE_C_FILES:$(CORE_SRC)/%.c=$(CORE_BIN)/%.o) $(CORE_CPP_FILES:$(CORE_SRC)/%.cpp=$(CORE_BIN)/%.o) 


# User library sources  (see) https://github.com/arduino/arduino/wiki/arduino-ide-1.5:-library-specification
LIB_DIRS_SHARED  := $(foreach d, $(LIBS_SHARED), $(LIBS_SHARED_BASE)/$d/ $(LIBS_SHARED_BASE)/$d/utility/)  # base and /utility 
LIB_DIRS_SHARED  += $(foreach d, $(LIBS_SHARED), $(LIBS_SHARED_BASE)/$d/src/ $(dir $(call rwildcard,$(LIBS_SHARED_BASE)/$d/src/,*/.)))                           # src and all subdirs of base

LIB_DIRS_LOCAL   := $(foreach d, $(LIBS_LOCAL), $(LIBS_LOCAL_BASE)/$d/ $(LIBS_LOCAL_BASE)/$d/utility/ )    # base and /utility  
LIB_DIRS_LOCAL   += $(foreach d, $(LIBS_LOCAL), $(LIBS_LOCAL_BASE)/$d/src/ $(dir $(call rwildcard,$(LIBS_LOCAL_BASE)/$d/src/,*/.)))                             # src and all subdirs of base

LIB_CPP_SHARED  := $(foreach d, $(LIB_DIRS_SHARED),$(call wildcard,$d*.cpp))
LIB_C_SHARED    := $(foreach d, $(LIB_DIRS_SHARED),$(call wildcard,$d*.c))

LIB_CPP_LOCAL   := $(foreach d, $(LIB_DIRS_LOCAL),$(call wildcard,$d*.cpp)) # $d already has trailing /
LIB_C_LOCAL     := $(foreach d, $(LIB_DIRS_LOCAL),$(call wildcard,$d*.c))   # $d already has trailing /

LIB_OBJ         := $(LIB_CPP_SHARED:$(LIBS_SHARED_BASE)/%.cpp=$(LIB_BIN)/%.o)  $(LIB_CPP_LOCAL:$(LIBS_LOCAL_BASE)/%.cpp=$(LIB_BIN)/%.o) 
LIB_OBJ         += $(LIB_C_SHARED:$(LIBS_SHARED_BASE)/%.c=$(LIB_BIN)/%.o)  $(LIB_C_LOCAL:$(LIBS_LOCAL_BASE)/%.c=$(LIB_BIN)/%.o) 

# Includes -------------------------------------------------------------
INCLUDE        := -I.\$(USR_SRC) -I$(CORE_SRC) 
INCLUDE        += $(foreach d, $(LIB_DIRS_SHARED), -I$d)
INCLUDE        += $(foreach d, $(LIB_DIRS_LOCAL), -I$d)

# Generate directories --------------------------------------------------------
DIRECTORIES :=  $(sort $(dir $(CORE_OBJ) $(USR_OBJ) $(LIB_OBJ)))
generateDirs := $(foreach d, $(DIRECTORIES), $(shell if [ ! -d "$(d)" ]; then mkdir -p "$(d)"; fi))

#$(info dirs: $(DIRECTORIES))

#$(info dirs: $(LIB_DIRS_LOCAL))
#$(info )
#$(info cpp: $(LIB_CPP_SHARED))
#$(info include: ${INCLUDE})

#******************************************************************************
# Rules:
#******************************************************************************

.PHONY: directories all rebuild upload uploadTy uploadCLI clean cleanUser cleanCore


all:  $(TARGET_LST) $(TARGET_HEX)

rebuild: cleanUser all

clean:   cleanUser cleanCore cleanLib

upload: all
	@$(UPL_PJRC)

uploadTy: all
	@$(UPL_TYCMD)

uploadCLI: all
	@$(UPL_CLICMD)


# Core library ----------------------------------------------------------------
$(CORE_BIN)/%.o: $(CORE_SRC)/%.S
	@echo CORE [ASM] $(notdir $<)
	@"$(CC)" $(S_FLAGS) $(INCLUDE) -o $@ -c $< 

$(CORE_BIN)/%.o: $(CORE_SRC)/%.c
	@echo CORE [CC]  $(notdir $<)
	@"$(CC)" $(C_FLAGS) $(INCLUDE) -o $@ -c $< 

$(CORE_BIN)/%.o: $(CORE_SRC)/%.cpp
	@echo CORE [CPP] $(notdir $<)
	@"$(CXX)" $(CPP_FLAGS) $(INCLUDE) -o $@ -c $< 

$(CORE_LIB) : $(CORE_OBJ)
	@echo CORE [AR]  $@
	@$(AR) $(AR_FLAGS) $@ $^
	@echo Teensy core built successfully && echo

# Shared Libraries ------------------------------------------------------------
$(LIB_BIN)/%.o: $(LIBS_SHARED_BASE)/%.S
	@echo LIB  [ASM] $(notdir $<)
	@"$(CC)" $(S_FLAGS) $(INCLUDE) -o $@ -c $< 

$(LIB_BIN)/%.o: $(LIBS_SHARED_BASE)/%.cpp 
	@echo LIB  [CPP] $(notdir $<)	
	@"$(CXX)" $(CPP_FLAGS) $(INCLUDE) -o $@ -c $< 

$(LIB_BIN)/%.o: $(LIBS_SHARED_BASE)/%.c
	@echo LIB  [CC]  $(notdir $<)	
	@"$(CC)" $(C_FLAGS) $(INCLUDE) -o $@ -c $< 
	
# Local Libraries -------------------------------------------------------------
$(LIB_BIN)/%.o: $(LIBS_LOCAL_BASE)/%.S
	@echo LIB  [ASM] $(notdir $<)
	@"$(CC)" $(S_FLAGS) $(INCLUDE) -o $@ -c $< 

$(LIB_BIN)/%.o: $(LIBS_LOCAL_BASE)/%.cpp
	@echo LIB  [CPP] $(notdir $<)	
	@"$(CXX)" $(CPP_FLAGS) $(INCLUDE) -o $@ -c $< 

$(LIB_BIN)/%.o: $(LIBS_LOCAL_BASE)/%.c
	@echo LIB  [CC]  $(notdir $<)	
	@"$(CC)" $(C_FLAGS) $(INCLUDE) -o $@ -c $< 
	
# Handle user sources ---------------------------------------------------------
$(USR_BIN)/%.o: $(USR_SRC)/%.S
	@echo USER [ASM] $<	
	@"$(CC)" $(S_FLAGS) $(INCLUDE) -o $@ -c $<

$(USR_BIN)/%.o: $(USR_SRC)/%.c
	@echo USER [CC]  $(notdir $<)	
	@"$(CC)" $(C_FLAGS) $(INCLUDE) -o "$@" -c $<

$(USR_BIN)/%.o: $(USR_SRC)/%.cpp
	@echo USER [CPP] $(notdir $<)	
	@"$(CXX)" $(CPP_FLAGS) $(INCLUDE) -o "$@" -c $<

# Linking ---------------------------------------------------------------------
$(TARGET_ELF): $(CORE_LIB) $(LIB_OBJ) $(USR_OBJ) 
	@echo [LD]  $@
	@$(CC) $(LD_FLAGS) -T$(CORE_SRC)/$(LD_SCRIPT) -o "$@" $(USR_OBJ) $(LIB_OBJ) $(CORE_LIB) $(LIBS)
	@echo User code built and linked to libraries && echo

%.lst: %.elf
	@echo [LST] $(notdir $@)
	@$(OBJDUMP) -d -S --demangle --no-show-raw-insn --syms "$<"  > "$@"
	@echo Listfile generated && echo

%.hex: %.elf
	@echo [HEX] $@
	@$(SIZE) "$<"
	@$(OBJCOPY) -O ihex -R.eeprom "$<" "$@"
	@echo Sucessfully built project && echo

# Cleaning --------------------------------------------------------------------
cleanUser:
	@echo Cleaning user binaries...
	@if [ -d $(USR_BIN) ]; then rm -r $(USR_BIN); fi


	@if [ -f $(TARGET_HEX) ]; then rm  $(TARGET_HEX); fi
	@if [ -f $(TARGET_ELF) ]; then rm  $(TARGET_ELF); fi
	@if [ -f $(TARGET_LST) ]; then rm  $(TARGET_LST); fi

cleanCore:
	@echo Cleaning core binaries...
	@if [ -d $(CORE_BIN) ]; then rm -r $(CORE_BIN); fi
	@if [ -f $(CORE_LIB) ]; then rm  $(CORE_LIB); fi

cleanLib:
	@echo Cleaning user library binaries...
	@if [ -d $(LIB_BIN) ]; then rm -r $(LIB_BIN); fi
	

# compiler generated dependency info ------------------------------------------
-include $(CORE_OBJ:.o=.d)
-include $(USR_OBJ:.o=.d)
-include $(LIB_OBJ:.o=.d)
Changes are ...

1) Most are obvious path and directory separator changes.
2) SHELL definition and export commented out – not required.
3) Shell commands changed to suit linux.
4) tyCommander changed to tycommander.
5) LIB_CPP_LOCAL and LIB_C_LOCAL – I removed an extra ‘/’ which was causing directory names such as ‘lib/SPI//’ to be produced. Windows apparently doesn’t mind this but linux did.
6) The echo commands in the build had an ‘echo.’ (note the ‘.’). Linux didn’t like that.
7) Clean commands changed to suit linux.


and the c_cpp_properties.json

Code: 
{
  "configurations": [
    {
      "name": "VisualTeensy",
      "includePath": [
        "src/**",
        "lib/**",
        "/home/hugh/arduino-1.8.7/hardware/teensy/avr/cores/teensy3/**",
        "/home/hugh/Arduino/libraries/**"
      ],
      "defines": [
        "__MK20DX256__",
        "TEENSYDUINO=144",
        "F_CPU=96000000",
        "USB_SERIAL",
        "LAYOUT_US_ENGLISH",
        "ARDUINO"
      ],
      "compilerPath": "/home/hugh/arduino-1.8.7/hardware/tools/arm/bin/arm-none-eabi-gcc",
      "intelliSenseMode": "gcc-x64"
    }
  ],
  "version": 4
}

and the tasks.json

Code: 
{
  "tasks": [
    {
      "label": "Build",
      "group": {
        "kind": "build",
        "isDefault": true
      },
      "command": "make",
      "args": [
        "all",
        "-j",
        "-Otarget"
      ]
    },
    {
      "label": "Rebuild User Code",
      "group": {
        "kind": "build",
        "isDefault": true
      },
      "command": "make",
      "args": [
        "rebuild",
        "-j",
        "-Otarget"
      ]
    },
    {
      "label": "Clean",
      "group": {
        "kind": "build",
        "isDefault": true
      },
      "command": "make",
      "args": [
        "clean"
      ]
    },
    {
      "label": "Upload (Teensy Uploader)",
      "group": {
        "kind": "build",
        "isDefault": true
      },
      "command": "make",
      "args": [
        "upload",
        "-j",
        "-Otarget"
      ]
    },
    {
      "label": "Upload (TyCommander)",
      "group": {
        "kind": "build",
        "isDefault": true
      },
      "command": "make",
      "args": [
        "uploadTy",
        "-j",
        "-Otarget"
      ]
    },
    {
      "label": "Upload (CLI)",
      "group": {
        "kind": "build",
        "isDefault": true
      },
      "command": "make",
      "args": [
        "uploadCLI",
        "-j",
        "-Otarget"
      ]
    }
  ],
  "version": "2.0.0",
  "type": "shell",
  "presentation": {
    "echo": true,
    "reveal": "always",
    "focus": false,
    "panel": "shared",
    "showReuseMessage": false
  },
  "problemMatcher": "$gcc"
}
 
Thanks a lot, hope that I can work on VisualTeensy next weekend. If you zip your *.rtf you can upload it directly to the forum (You need to switch to the advanced view first)
 
Status
Not open for further replies.
Back
Top