Command line only compile (not upload) with arduino-builder

J

jamesportman

Member
Hi,

My goal is to run a compilation only test (think CI/CD),
I have searched and found lots of info but can't figure this out quite.

I am trying to use arduino-builder, it does work but for whatever reason it really wants to open the gui version of teensy loader, even when not carrying out an upload.
This works fine on normal PC but it won't inside docker or on a ci server.

I have seen arduino-cli but that also wants to use the gui teensy loader.

Is there a way to stop arduino-builder from trying to run the gui app?

Code: 
ARDUINO_DIR="/home/james/bin/arduino-1.8.13-linux64/arduino-1.8.13"

TMP_DIR="$(mktemp -d)"

"${ARDUINO_DIR}/arduino-builder" \
-hardware "${ARDUINO_DIR}/hardware" \
-tools "${ARDUINO_DIR}/tools-builder" \
-tools "${ARDUINO_DIR}/hardware/tools/avr" \
-tools "${ARDUINO_DIR}/hardware/tools/teensy" \
-libraries "${ARDUINO_DIR}/libraries" \
-build-path "${TMP_DIR}" \
-fqbn "teensy:avr:teensy41:usb=serial,speed=600,opt=o2std,keys=en-gb" \
aprogram.ino
 
It is set up to open the Teensy Loader to pass the Name of the HEX file for a button push upload, even on just a Verify build with no upload.

If Windows was in use the TSET tool works for COmmand Line builds using this line to do the build without triggering the Loader to open?

Code: 
"%arduino%\arduino-builder" -verbose=1 -warnings=more -compile -logger=human -hardware "%arduino%\hardware" -hardware "%LOCALAPPDATA%\Arduino15\packages" -tools "%arduino%\tools-builder" -tools "%arduino%\hardware\tools\avr" -tools "%LOCALAPPDATA%\Arduino15\packages" -built-in-libraries "%arduino%\libraries" -libraries "%libs%" -fqbn=%fqbn% -build-path %temp1% -build-cache "%temp2%"  %ino%

Maybe the difference will pop out?
Perhaps the "-compile"
 
Thanks, I just tried with compile and a few other options, it still tries to open the loader.

I just realised that it does actually work in general (compiles) but then complains about not being able to open the teensy loader which was making my script exit with an error level set.
The hex file etc is all there in the output which is good, but not helpful because I can't really distinguish between compilation failure or it just moaning about the loader.

I guess for now I can make sure the build path is empty before running and look for the hex file to see if it worked?
 
The cmdline build still runs the IDE .txt files - and that invokes the standard build. On Windows above line and process here doesn't invoke loader as used - maybe linux processes differently?

TSET has a next line here that invokes loader or TyCommander selectively when a 'Build with Upload' versus 'Verify' build is selected.
 
Have you tried building with recent versions of Arduino-CLI and Teensy installs?

I thought that Paul put something in that detected if running on a GUI and if not runs without gui...
 
arduino-cli was the first thing I tried but it wasn't working, I need to try it again really.
I did try newer versions of arduino IDE as well but they were segfaulting just from arduino-builder runs.. so that's why I'm on 1.8.13

I found a solution for now, I ran strace to find out what was actually being run, it was teensy_post_compile that ended up running the loader.
hardware/teensy/avr/platform.txt
65 ## Post Build - inform Teensy Loader of new file
66 recipe.hooks.postbuild.1.pattern="{teensytools.path}teensy_post_compile" "-file={build.project_name}" "-path={build.path}" "-tools={teensytools.path}" "-board={build.board}"

Commenting that line out fixes it for me.
I am running in docker so I'll just do that for now and test arduino-cli in a couple of weeks?


Edit: confused about why you would trigger the loader on a build and not just for uploads, is there a reason?
There is the above hook for all builds, then there is also another line for when user selects upload.
106 tools.teensyloader.upload.pattern="{cmd.path}/teensy_post_compile" "-file={build.project_name}" "-path={build.path}" "-tools={cmd.path}" "-board={build.board}" -reboot "-port={serial.port}" "-portlabel={serial.port.label}" "-portproto col={serial.port.protocol}"
I'm not understanding what the point of the post build hook is.
 
Last edited:
Yeah, the post compile step is what I was guessing at in the .txt files from the IDE.
Somehow the TSET line presented skips that when the version to defer to TyCommander worked to exclude that ...
 
Sorry edited previous post while you were typing.
Not sure what the point of the post build hook is, if it doesn't actually end up doing anything.
 
jamesportman said:
Sorry edited previous post while you were typing.
Not sure what the point of the post build hook is, if it doesn't actually end up doing anything.
Click to expand...

See p#2.
IDE supports a Verify build that results in a usable HEX that is not uploaded.

Since Teensy uses the External loader it is started and passed the path to the HEX.

This allows first programming where a shipped Teensy has no active USB for Auto Upload ... or in any similar case where Teensy built with no USB, or where crashed code disables USB.

When in this state with Loader knowing HEX and the Button is pressed the Loader sees the Teensy go into Bootloader mode and then responds with an upload.
 
Right thanks.
I did read p2 but the reasoning wasn't there.
If it is needed when the board might be in some strange state then sure, fair enough.
 
Teensy has always supported 2 ways to program, either automatic upload or compile & press the button.

The compile & press button approach serves as a fallback for cases were automatic upload fails. While it's not common with "normal" programming, your program can crash or go wrong in ways that disrupt USB communication, which prevents Teensy from hearing and responding to the request your PC sends to initiate it automatically.

But things haven't always worked as smoothly over Teensy's 14 year history as they (mostly) do today. About half of those years were before Windows 10 (and the first few years had most Windows enthusiasts clinging to Windows 7). All pre-10 versions of Windows had rather buggy USB drivers. Windows 7 in particular had lengthy and strange delays when detecting USB devices, during which time is would keep a dialog on the screen, and if the user gave up or was just impatient (eg, me always) and canceled, things would often go badly and sometime even with corrupted junk left in the Windows register. Before Windows 10, Microsoft didn't ship an INF file to cause their own (then quite buggy) USBSER.SYS driver to load, which also caused a lot of trouble for Windows users. Since the very beginning, Teensy has used HID protocol rather than serial because Windows didn't have an INF but does have HID drivers. At some point we added RawHID and started shipping every Teensy with LED blink compiled as RawHID rather than serial, because some Windows users in those days would plug Teensy into their PC before installing the software and then upon seeing the notification from Windows that a driver was needed they would pretty much go to the end of the earth searching online and trying to install all sorts of random stuff (but not run the Teensyduino installer) and some even dove into regedit and damaged their Windows installs. It really was a tech support nightmare in those early days. Today with Windows 10 these "driver problems" are mostly a distant memory, but for the first several years of Teensy (long before Arduino made Leonardo, their first native USB board) this was a regular issue and the method of compile and press the pushbutton was the solution that always worked.

Linux and MacOS also changed over time. In the early days many Linux distros were migrating from "hotplug" to udev, and udev shared libraries has a major version (incompatible with previously compiled software) change. Apple dragged their feet for a few years, until MacOS 10.7 released in 2011, before supporting CDC serial as part of composite device descriptors. Windows had most of the problems, but early MacOS and Linux were far from what we take for granted today. I personally found and reported issues in all 3 systems (thanks to Phil T at Adafruit for making the contact within Microsoft), which were eventually fixed.

So even though today the verify & press button approach is only needed on fairly rare occasions (but when it's needed, you *really* need it) the design of this feature Teensy is rooted in the bad-old-days before Windows 10, MacOS 10.7 and Ubuntu 10.04 when USB support greatly improved.
 
Last edited:
Hi Paul,

You didn't have to explain yourself but thanks for the info, it is interesting.
Maybe not a teensy but I definitely got an arduino of some sort completely stuck rebooting in the past and it was very difficult to program it again.
The backup method of programming the teensy does make sense, and yes I bet you are really glad it's there when you have an almost bricked device.

I have seen a few things like printers with stickers over the USB ports explaining that you really must install the bundled software before plugging them in.
I bet it is hard work being on the support side of things like you are.

I'm happy with the current solution, just took a bit of time to figure out.
Hopefully if someone else is stuck with the same issue then they might find this.
 
You must log in or register to reply here.
Back
Top