Arduino CLI And IDE now Released - Teensy Supported!

[PaulStoffregen]

I got the multithreaded port discovery tool running with defines to translate pthreads functions to WIN32, and I'm able to receive the WM_DEVICECHANGE messages on a specific thread. Should be able to get everything updated to MacOS and Windows tomorrow.

 

[mjs513]

I got the multithreaded port discovery tool running with defines to translate pthreads functions to WIN32, and I'm able to receive the WM_DEVICECHANGE messages on a specific thread. Should be able to get everything updated to MacOS and Windows tomorrow.

That's great news Paul - looking forward to playing with the new IDE been holding off until you give us the word. But not sure if IDE 2.0 is all that much better than the current version but will hold my reservations until after we start using it.

 

[PaulStoffregen]

Still working on porting to MacOS & Windows. Turns out I was thinking about each system's threads and hardware detection differ, but not so much about the differences in how each system's API differ between HID, serial, localhost network. Had to go back and restructure the multithreaded serial monitor tool to pass abstract structures rather than hard coding Linux "everything is an integer file descriptor" throughout.

Will restart my MacOS and Windows effort later today. Pretty sure it will go better this time.

 

[mjs513]

Still working on porting to MacOS & Windows. Turns out I was thinking about each system's threads and hardware detection differ, but not so much about the differences in how each system's API differ between HID, serial, localhost network. Had to go back and restructure the multithreaded serial monitor tool to pass abstract structures rather than hard coding Linux "everything is an integer file descriptor" throughout.

Will restart my MacOS and Windows effort later today. Pretty sure it will go better this time.

Think we all have been there. As you said goes easier on the second round. Good luck with the updates

 

[KurtE]

Think we all have been there. As you said goes easier on the second round. Good luck with the updates

Who me?

Paul - Definitely been there

All - semi random

Was playing around some with Linux box and was wondering if I could setup like when I am working on some of the Robotis boards like OpenCM or OpenCR, where I have their boards listing under tools->boards twice, one for the current release and another time for the DEV branch... (they work a little different in that their master/main branch is the current released code, and the dev branch is the WIP.

So I hacked up a setup, where I created the directory ~/Arduino/hardware/teensy_189
And then I created a symbolic link to the stuff I am building using arduino 1.8.9, the directory I created: ln -s ~arduino-1.8.19/hardware/teensy/avr

Then they both show up in the tools hardware menu:


However the build won't work...

Compilation error: fork/exec /home/kurte/Arduino/hardware/teensy_189/../tools/arm/bin/arm-none-eabi-g++: no such file or directory

The problem is the system can not find the build tools as the platform.txt is setup to hard coded ../tools ...

Paul, I know you already know this as you updated the platform.txt for the arduino cli install to handle this:

# Teensyduino Installer
#compiler.path={runtime.hardware.path}/../tools/
#teensytools.path={runtime.hardware.path}/../tools/

# Arduino Boards Manager
compiler.path={runtime.tools.teensy-compile.path}/
teensytools.path={runtime.tools.teensy-tools.path}/

By changing the platform.txt to not use the relative names, but instead where the tools are installed in the ~/.arduino/packages/teensy

Again I know this is premature, but looks like this type of an approach could work....

 

[PaulStoffregen]

I've updated package_teensy_index.json for MacOS. So far I've only tested with arduino-cli, but in theory it ought to also work with Arduino 2.0-rc3.

The new serial monitor support has a known issue where it does not properly close the port. I will fix this after porting to Windows. Figuring out how to properly close the port is complicated because it's used by 2 threads (each moves data in 1 direction).

And yeah, platform.txt used with Arduino 1.8 uses paths relative to {runtime.hardware.path}, but that isn't supported by arduino-cli and Arduino 2.0.

 

[KurtE]

I've updated package_teensy_index.json for MacOS. So far I've only tested with arduino-cli, but in theory it ought to also work with Arduino 2.0-rc3.

The new serial monitor support has a known issue where it does not properly close the port. I will fix this after porting to Windows. Figuring out how to properly close the port is complicated because it's used by 2 threads (each moves data in 1 direction).

And yeah, platform.txt used with Arduino 1.8 uses paths relative to {runtime.hardware.path}, but that isn't supported by arduino-cli and Arduino 2.0.

Tried it using nightly build on my old mac... Installed,

Tried to build blink for T4.1 and the ld failed...

Collect2: error: ld returned 1 exit status

 

[PaulStoffregen]

Today I'm working on porting the multithreaded serial monitor tool to Windows. Just now got the code to compile.

Looks like Arduino merged #1640. Probably tomorrow we'll have a nightly build that gives the upload process enough info to tell when to use GUI vs headless.

Not sure what's up with the MacOS linker. Will look at that after I get Windows communicating.

 

[PaulStoffregen]

Well, getting pluggable monitor working on Windows with Arduino CLI took longer than I wanted.



Want to spend a little time actually running several tests (to avoid a repeat of my MacOS issues) and then package it up for Window and redo the Mac package. Should be later today...

 

[mjs513]

Well, getting pluggable monitor working on Windows with Arduino CLI took longer than I wanted.

....

Want to spend a little time actually running several tests (to avoid a repeat of my MacOS issues) and then package it up for Window and redo the Mac package. Should be later today...

Great you got it working on Windows. Take your time - found if I rush, I wind up making more mistakes. Looking forward to playing.

 

[mjs513]

Well, getting pluggable monitor working on Windows with Arduino CLI took longer than I wanted.


Want to spend a little time actually running several tests (to avoid a repeat of my MacOS issues) and then package it up for Window and redo the Mac package. Should be later today...

I know this is very premature, but I went ahead and installed the nightly build from this morning and added the TeensyJson to the additional boards manager URL list and everything installed very nicely including libraries. So decided to run a simple blink sketch (just a compile) for Teensy 41 and TMM, and getting this compile error:

c:/users/merli/appdata/local/arduino15/packages/teensy/tools/teensy-compile/1.56/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld.exe: cannot find -larm_cortexM7lfsp_math
collect2.exe: error: ld returned 1 exit status
Compilation error: exit status 1

Blink on the T3.5 gives this error:

c:/users/merli/appdata/local/arduino15/packages/teensy/tools/teensy-compile/1.56/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld.exe: cannot find -larm_cortexM4lf_math
collect2.exe: error: ld returned 1 exit status
Compilation error: exit status 1

 

[KurtE]

Note: I downloaded today's nightly build and one thing I noticed is Norton is giving me lots of warnings and blocked: mdns-discovery.exe
which I am guessing is something with this IDE?

 

[KurtE]

I know this is very premature, but I went ahead and installed the nightly build from this morning and added the TeensyJson to the additional boards manager URL list and everything installed very nicely including libraries. So decided to run a simple blink sketch (just a compile) for Teensy 41 and TMM, and getting this compile error:

c:/users/merli/appdata/local/arduino15/packages/teensy/tools/teensy-compile/1.56/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld.exe: cannot find -larm_cortexM7lfsp_math
collect2.exe: error: ld returned 1 exit status
Compilation error: exit status 1

Blink on the T3.5 gives this error:

c:/users/merli/appdata/local/arduino15/packages/teensy/tools/teensy-compile/1.56/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld.exe: cannot find -larm_cortexM4lf_math
collect2.exe: error: ld returned 1 exit status
Compilation error: exit status 1

Oops :0 - I decided to jump in as well.

I think I know the issue (or maybe just first on)
If I look at the libraries installed on 1.8.19 with current TD versus what was installed it is missing 4 lib files:

Will copy those over and see...

Update: I did it and now I can build and install blink sketch on MMOD...

Will go back to Mac and see if I can do the same.

Update 2: copy those 4 files on Mac worked ...
Took me awhile to find where Arduino15 is on the MAC

 

[mjs513]

Oops :0 - I decided to jump in as well.

I think I know the issue (or maybe just first on)
If I look at the libraries installed on 1.8.19 with current TD versus what was installed it is missing 4 lib files:
...
Will copy those over and see...

Update: I did it and now I can build and install blink sketch on MMOD...

Will go back to Mac and see if I can do the same.

Update 2: copy those 4 files on Mac worked ...
Took me awhile to find where Arduino15 is on the MAC

Sorry been out of it most of the day but I just copied those files as you did on a Windows machine and so far all sketches compile without a linker error. Teensloader also opens but not finding ports yet.

Update 1: As a test I compiled my MTP test sketch and got it to upload by pressing the program button on the Micromod breakout board. Then used Tycommander as my Serial Monitor and it worked fine. Out put from IDE 2.0

Opening Teensy Loader...
teensy_size: Memory Usage on Teensy MicroMod:
teensy_size:   FLASH: code:159340, data:25928, headers:8264   free for files:16321540
teensy_size:    RAM1: variables:50848, code:150824, padding:13016   free for local variables:309600
teensy_size:    RAM2: variables:17504  free for malloc/new:506784

--------------------------
Compilation complete.
Upload error: Failed uploading: no upload port provided

Tycommander output:

Dump Storage list(6)
store:0 storage:10001 name:sdio fs:2000becc pn:
store:1 storage:20001 name:PROGM fs:2000bda0 pn:PROGRAM
store:2 storage:30001 name:sflash5 fs:2000c39c pn:W25Q512JV*M (DTR)
store:3 storage:40001 name:sflash6 fs:2000c478 pn:W25Q512JV*Q
store:4 storage:50001 name:WINBOND1G fs:2000b954 pn:W25N01GVZEIG
store:5 storage:60001 name:WINBOND2G fs:2000ba48 pn:W25N02KVZEIR

 

[defragster]

Cool - downloaded and unzipped the new current arduino-ide_nightly-20220201_Windows_64bit ...

Hope that version good to go when the beta drops.

With PJRC JSON added I see IDE 1.8.19 board manager finds TEENSY ... wonder if it works the same as for IDE 2.0? Other boards seem to transfer functionally.

 

[KurtE]

I also have had some luck uploading sketches.

Sometimes the com port menu is there other times not. Sometime when not, if I exit the IDE it will appear than.

Like Ubuntu does not work with the Arduino IDE serial monitor

 

[PaulStoffregen]

I updated package_teensy_index.json with tools for Windows. Replaced the compiler tool with one that has the missing libraries, and fat trimmed from AVR toolchain for Teensy 2.0.

So far all my testing has been with arduino-cli. Will start trying IDE 2.0 soon. With arduino-cli, I'm seeing at least one bug where pluggable monitor seems to be intermittently failing to associate the correct protocol. Not sure if that will affect the IDE's serial monitor.

 

[defragster]

Paul - when you say arduino-cli is that referring to installing with current IDE 1.8.19 family?

Using Board Manager there no and it is installing ... taking some time to deliver to %appdata% ...

TeensyInstaller was MCUH nicer! And easily portable/allowing multiple variant/version installs.

Built in IDE 1.8.19 here to Encrypted T_4.1 - build paths (tools and libs) suggest it is using the 'packages 1.56.1' version - nothing from TeensyInstaller folder:

...
"C:\\Users\\Tim\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.56.1/teensy_secure" encrypthex TEENSY41 "R:\\Temp\\arduino_build_495784/MakeFiles.ino.hex"
encrypting 91136 bytes to R:\Temp\arduino_build_495784/MakeFiles.ino.ehex
...
Multiple libraries were found for "SD.h"
 Used: C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.56.1\libraries\SD
 Not used: C:\T_Drive\arduino-1.8.19\libraries\SD
Using library SD at version 2.0.0 in folder: C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.56.1\libraries\SD 
Using library SdFat at version 2.1.0 in folder: C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.56.1\libraries\SdFat 
Using library SPI at version 1.0 in folder: C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.56.1\libraries\SPI 
C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\1.56.1/teensy_post_compile -file=MakeFiles.ino -path=R:\Temp\arduino_build_495784 -tools=C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\1.56.1 -board=TEENSY41 -reboot -port=usb:0/140000/0/6/3 -portlabel=COM8 Everything -portprotocol=Teensy

Ports shoed the 'Teensy' Variant and uplaoded.
Opened Teensy Loader and uploaded no issue.

Edit similar results with IDE 2.b3_Feb01 - except the port finding, it did upload.

...
Multiple libraries were found for "SD.h"
 Used: C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.56.1\libraries\SD
 Not used: C:\T_Drive\arduino-1.8.19\libraries\SD
Using library SD at version 2.0.0 in folder: C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.56.1\libraries\SD 
Using library SdFat at version 2.1.0 in folder: C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.56.1\libraries\SdFat 
Using library SPI at version 1.0 in folder: C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.56.1\libraries\SPI 

--------------------------
Compilation complete.
"C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\1.56.1/teensy_post_compile" "-file=MakeFiles.ino" "-path=R:\TEMP\arduino-sketch-D6B7C4F3A9B1FEE3AECA7BEEFFDCBB17" "-tools=C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\1.56.1" "-board=TEENSY41" -reboot "-port=COM11" "-portlabel={serial.port.label}" "-portprotocol={serial.port.protocol}"

 

[PaulStoffregen]

Paul - when you say arduino-cli is that referring to installing with current IDE 1.8.19 family?

arduino-cli is a command line version. You can download it here:

https://arduino.github.io/arduino-cli/0.20/installation/

Supposedly the new IDE does everything by using arduino-cli.

 

[defragster]

arduino-cli is a command line version. You can download it here:

https://arduino.github.io/arduino-cli/0.20/installation/

Supposedly the new IDE does everything by using arduino-cli.

Okay, wanted to be sure I was following - independent but underlying the IDE's

... as IDE 1.8.19 of course effectively builds now with a CLI - used by TSET from a DOS box. So just morphing to an alternate updated CLI to support the new IDE 2.0 - where they are using the existing %appdata%/packages format as maintained by Board and Library Manager interface.

 

[mjs513]

I updated package_teensy_index.json with tools for Windows. Replaced the compiler tool with one that has the missing libraries, and fat trimmed from AVR toolchain for Teensy 2.0.

So far all my testing has been with arduino-cli. Will start trying IDE 2.0 soon. With arduino-cli, I'm seeing at least one bug where pluggable monitor seems to be intermittently failing to associate the correct protocol. Not sure if that will affect the IDE's serial monitor.

Couldn't sleep again so decided to play some more. I noticed with the update that boards manage showed TD version 1.56.1. On clicking install 1.56.1 output showed that it uninstalled old version and installed the updated version. On verify sketch no linker errors were seen for BLINK.

This time around the TMM port was found so was able to do an upload. However, the IDE was silent on what was going on (no progress bar) unless I selected verbose in preferrences which was rather annoying but the upload was successful.

Using the MTP Test sketch next, after clicking the upload button the board/port combo failed to connect via serial monitor so have to use TyCommander:


Been trying to add in @KurtE's MTP+Serial configuration which we have been using with success but when we add a boads.local.txt file it doesn't seem to recognize the file or added USB Type. Even if I add it into the existing boards.txt file it doesn't pick it up. For that matter even if I delete MTP Disk it still shows in the menu - so not sure whats going on????? Any suggestions.

Update 1. in going through this issue: https://github.com/arduino/arduino-cli/issues/1614 saw that they deleted the following directory first after adding in boards.local.txt

C:\Users\<user name>\AppData\Roaming\arduino-ide\

Did that and then it picked up the boards.local.txt file

Update 2: Recompiling for MTP+Serial Serial Monitor now shows data only if I select the actual com port and not the Everything port that we usually select. Data is received from the TMM but can't seem to transmit back to the TMM:

 

[PaulStoffregen]

On clicking install 1.56.1 output showed that it uninstalled old version and installed the updated version.

Yes, I added a ".1" to the version number, so it would be easier to see if it's using the updated tools from yesterday or my original botched attempt from the weekend.

There was also a period of several hours yesterday where non-working Windows files (but not the stuff from the weekend) identified at 1.56.0 were on the server as a sort of placeholder to help me get my Windows test system set up. So adding the ".1" when I finally put a set of (hopefully) good files on the server seemed like a good idea.

Okay, wanted to be sure I was following - independent but underlying the IDE's

Yes, pretty much.

I believe the concept is to provide scripts (like used with github actions to automatically check code) with a full set of the Arduino IDE functionality.

... as IDE 1.8.19 of course effectively builds now with a CLI - used by TSET from a DOS box. So just morphing to an alternate updated CLI to support the new IDE 2.0

Difficult to know quite what you mean by this. If your concept is a batch file or script which runs the compile process, yes, it certainly can do that.

But Arduino CLI has a lot more functionality than just compiling. Pretty much all the operations the Arduino 2.0 IDE actually does (eg, managing boards, installing libraries, etc) are performed by Arduino CLI.

It also has a server mode where it accepts commands over a localhost TCP socket, rather than as command line input.

 

[defragster]

...

Difficult to know quite what you mean by this. If your concept is a batch file or script which runs the compile process, yes, it certainly can do that.

But Arduino CLI has a lot more functionality than just compiling. Pretty much all the operations the Arduino 2.0 IDE actually does (eg, managing boards, installing libraries, etc) are performed by Arduino CLI.

It also has a server mode where it accepts commands over a localhost TCP socket, rather than as command line input.

Seems you got the gist ... yes with 1.8.19 and prior TSET [3+ years in use] uses a batch file to do the build - after running a quick batch file that sets the board options and creates it.

And the new morphed CLI does all that and more. Frank found the old 'arduino-builder' line in the console, hopefully I can find that exposed in the IDE 2.0 and replicate its passing of the Board params.

 

[PaulStoffregen]

Just spent some time with the new IDE. Looks like it's using pluggable discovery but isn't using pluggable monitor.

 

[mjs513]

Yes, I added a ".1" to the version number, so it would be easier to see if it's using the updated tools from yesterday or my original botched attempt from the weekend.

There was also a period of several hours yesterday where non-working Windows files (but not the stuff from the weekend) identified at 1.56.0 were on the server as a sort of placeholder to help me get my Windows test system set up. So adding the ".1" when I finally put a set of (hopefully) good files on the server seemed like a good idea.

Was actually a great idea - made updating a lot easier, otherwise I would have had to uninstall Teensy and then reinstall.

Know this is still a work in progress but just as you might say things noticed/tried list:
1. Noticed Teensy "Built-in examples", did not appear on the examples menu
2. So the crazy person that I am I copied over the triple serial example and ran it for the TMM. Kind of sort of worked - had to press the PGM button to make it upload. All three ports did show up but I had to specifically assign TMM to each port. Also without the ability to send from the serial monitor couldn't really test.