Arduino CLI And IDE 2 RC Release - Teensy Support?

[PaulStoffregen]

Is configuring your temp folder on a normal drive an option?

{Windows rant mode} The problems I just fixed in teensy-monitor.exe turned out to be WIN32 ReadFile & WriteFile don't work together when called from separate threads when configured in the simpler blocking mode, which is the way you would naturally want to use them on separate threads. Normally you'd only want "overlapped" async I/O without threads, where blocking on anything stalls everything else. Seems as if Microsoft implemented the blocking API with a mutex, making it useless for multiple threads. Turns out you must use the more complicated "overlapped" async API if you want WriteFile running in 1 thread to be able to actually transmit while ReadFile is waiting in another thread for data to arrive (and when it returns with data, the waiting WriteFile finally gets to transmit - which is the reason Kurt's test program works 5 seconds later after Teensy transmit anything). So many things in Windows are like this, where Microsoft gives you a supposedly generic API like CreateFile, ReadFile, WriteFile, but then when you actually use it lots of things don't work except in some specific mode. While you must use overlapped on serial and HID with threads where you just want each thread to block, turns out you can't use async I/O at all on stdin & stdout. The teensy-serialmon.exe program for Arduino 1.8.x creates 2 extra threads just to work around the lack of async I/O for stdin & stdout, but then does everything using overlapped async I/O from a single thread.

I guess it's not as bad as the MacOS kernel panic triggered by certain usages of their async I/O API, which was the main motivation to move away from a single thread model.

Virtual drives having incomplete or inconsistent filesystem API support just so sees on-brand for Windows. {yeah, I'm still frustrated by ReadFile blocking WriteFile on another thread, even though it's now "fixed" by using overlapped mode}

[KurtE]

Is configuring your temp folder on a normal drive an option?

{Windows rant mode}

It is funny, I rant about every time I do something on Linux

But it looks like @defragster's issue report, does appear to be an issue that Arduino code currently has with Windows 11 using temp on Virtual drive.

Right now hacking on some code, I am curious if our code SD code works OK if I do a new SDClass()... delete....
So hacking up sketch with RC9...

And have several compiler errors. But I am also setup with an "Experimental error navigation" option that who knows if it will survive into release, but is interesting that it allows at least some way to navigate between errors.

To turn on: hit F1 to bring up a command list (Not sure what they call it).
I typed settings: and choose: Preferences: Open Settings (JSON)

And add the line: "arduino.compile.experimental": true

Save the settings and on the compile:

And the lines with errors will be marked and the current one has a previous/next
that you can click on as you can see in my quick and dirty:

[defragster]

Is configuring your temp folder on a normal drive an option?
...
Virtual drives having incomplete or inconsistent filesystem API support just so sees on-brand for Windows. {yeah, I'm still frustrated by ReadFile blocking WriteFile on another thread, even though it's now "fixed" by using overlapped mode}

No, RamDrive configuration is not normal.
> Other User noted this 25 days ago on Windows 10, where like here it worked in prior IDE 2 nightly versions tried in Jan/Feb
> This shows then as IDE 2 change and loss of functionality, not a failure of the RamDrive to provide full support

Done here because:
1> with 32 GB of RAM the system doesn't show even 16GB in use most of the time, 6GB RAM is put to good use.
2> Spastic TEMP usage would lead to SSD drive excess wear with soiling of FLASH blocks
3> CrystalDiskMark works to test the disk - and shows PERF is 2 to 10 times faster
4> it was fun and useful at times in MS-DOS days

[defragster]

It is funny, I rant about every time I do something on Linux

But it looks like @defragster's issue report, does appear to be an issue that Arduino code currently has with Windows 11 using temp on Virtual drive.

Right now hacking on some code, I am curious if our code SD code works OK if I do a new SDClass()... delete....
So hacking up sketch with RC9...

And have several compiler errors. But I am also setup with an "Experimental error navigation" option that who knows if it will survive into release, but is interesting that it allows at least some way to navigate between errors.
...
And the lines with errors will be marked and the current one has a previous/next
...

Indeed, VirtualDisk/RamDrive fail seems an IDE 2 regression that was not an issue in releases a month+ back.
> I apparently wrote a decent issue with details as it was a DUPE, but the prior issue was the one closed
> After posting did some edits/efforts to ID or 'work around' changing the drive creation that did not resolve.

SubLimeText is cool having "F4" next error navigation - hopefully it survives into IDE 2 release if it will 'be all it can be'

[PaulStoffregen]

Somewhat offtopic, but... I would have imagined a ramdisk on a modern PC would be a lot faster for random reads than only 222 MB/sec. That's a 4K read latency of approx 18 microseconds. 18 us is a low lower than NAND flash, but not even the 10 us number Intel claims for Optane SSD (eg, model 905P or P5800X).

[defragster]

Somewhat offtopic, but... I would have imagined a ramdisk on a modern PC would be a lot faster for random reads than only 222 MB/sec. That's a 4K read latency of approx 18 microseconds. 18 us is a low lower than NAND flash, but not even the 10 us number Intel claims for Optane SSD (eg, model 905P or P5800X).

Indeed. Perhaps system overhead, or that interface to the virtual drive just isn't optimal, or a problem in the measuring? Given the SEQ 2X-4X read boost and 10X+ write boost over an SSD it is unclear why RND improves so little over the non-Pro Gen3 NVMe SSD in use.

This review {thanks RaymondC} raymond.cc/blog/12-ram-disk-software-benchmarked-for-fastest-read-and-write-speed/ looks into 11 not 12 options.
> that trend persists in the CrystalMark #'s - system is an AMD not Intel CPU. Drive Params may be diff - but seems faster than this system.
His summary suggests OSFMount as perhaps best overall. I'll give that a try as it turns out both of 'us' with IDE 2 problems are using the same IMDisk.

> New machine last year I settled for Gen 10 i7 (before Gen 11) making sure chip shortage didn't leave my high and dry - but Gen 11 perf up maybe 40% and new Gen 12 are 95% faster CPUs, and now support NVMe Gen4 that may double throughput.

[defragster]

Updated issue on IDE at github. Swapped to OSFMount {it doesn't have a start on boot } so cannot use that as a replacement
> The OSFMount is a PASSMARK tool - allows mounting any image file for testing purposes

Using that code instead of IMDisk had same result for RC9: Fail to start

Returning to unzip of OLD Jan 28 nightly showed it to WORK now as it did then, starts and asks for permission and installs and brings up the usable IDE. So it is a new bug or 'regression' from old working behavior.

RE Passmark, they have Benchmark programs and list of all CPU's and a compare tool like this for Gen 10, 11, 12 i7's

[KurtE]

Quick note: Looks like RC9.1 just was released

Downloading now

[joepasquariello]

Quick note: Looks like RC9.1 just was released

Installed IDE 2.0 for the first time last week (rc9) on Windows 7 64-bit. Today when I started IDE 2.0, it notified me that rc9.1 was available. Download/install was fully automatic, so that was pretty impressive. I can see that build times are clearly faster with 2.0 than with 1.x.

[defragster]

Quick note: Looks like RC9.1 just was released

Downloading now

Doesn't look like my desktop will run it yet. Only activity on issue is: arduino-ide/issues/1236

changed the title RC9 fails to open, stalls at white window and busy spinner RC9 fails to open on a RAM drive, stalls at white window and busy spinner 6 hours ago

[defragster]

Installed IDE 2.0 for the first time last week (rc9) on Windows 7 64-bit. Today when I started IDE 2.0, it notified me that rc9.1 was available. Download/install was fully automatic, so that was pretty impressive. I can see that build times are clearly faster with 2.0 than with 1.x.

I started with ZIP install and saw the Auto Update work as well - very cool, except for Update it then runs the MSI Installer leaving the ZIP install orphaned and now running form the Arduino install location.

[PaulStoffregen]

I can see that build times are clearly faster with 2.0 than with 1.x.

I've been considering bringing recent Arduino CLI into the Teensyduino installer for Arduino 1.8.19, so we can make use of the improved caching features.

[KurtE]

I've been considering bringing recent Arduino CLI into the Teensyduino installer for Arduino 1.8.19, so we can make use of the improved caching features.

Might be good, but to be honest, I have not been using the 1.8.19 IDE very much these days. IDE2 is working a lot nicer now. Still have it's warts, especially with the Serial Monitor... But in that case if there is an issue, I then switch to using
TyCommander for that part.

[joepasquariello]

I've been considering bringing recent Arduino CLI into the Teensyduino installer for Arduino 1.8.19, so we can make use of the improved caching features.

I think that's a good idea. Nice to have the option of using the stable IDE 1.x, but get the faster build times, while IDE 2.0 is maturing.

[PaulStoffregen]

Many people probably won't touch IDE 2 until Arduino calls it officially released.

[mjs513]

Many people probably won't touch IDE 2 until Arduino calls it officially released.

Just finished downloading RC9.1 and had to give the Compiling progress bar a test. Really liked that they added that feature - makes it easier to see whats going on.

Release of IDE2 has to be getting closer since this is the first time I have seen release a RCx.1 type of release. Wondering how much longer before formal release of IDE2 since as Kurt said

IDE2 is working a lot nicer now. Still have it's warts, especially with the Serial Monitor...

and most of major bugs seem to be worked out?

[KurtE]

Just finished downloading RC9.1 and had to give the Compiling progress bar a test. Really liked that they added that feature - makes it easier to see whats going on.

Yes - I had not noticed that yesterday, but it does help to let people know something is going on, especially if they have not turned-on verbose output for compiles.

Release of IDE2 has to be getting closer since this is the first time I have seen release a RCx.1 type of release. Wondering how much longer before formal release of IDE2 since as Kurt said
and most of major bugs seem to be worked out?

As I mentioned I wish they would fix a few of the Serial Monitor issues, like the last line is clipped. Also selecting text and pasting and the keyboard selection and the like.
Also @defragsters setup with TEMP on Ram drive. would be nice.

Also, along the lines that this one is 9.1 instead of 10, I also noticed that when you look up at issues on github, they no longer show milestones.

So maybe fingers crossed, they may be getting closer to an official release.

[ptalbot]

When I read the different threads about the Teensy 4.1, Arduino and arduino-cli, I find a lot of information but nothing that solve the problems on my side so I might be missing something.

My environment is under Raspberry pi 4 with a Teensy 4.1 connected on the USB port

In the Arduino IDE 1.8.19, I see the tools/port: /dev/bus/usb/001/004 Bootleader (Teensy 4.1). The preferences.txt has the following line boardsmanager.additional.urls=https://www.pjrc.com/teensy/td_156/package_teensy_index.json
When I do Tools/Get Board Info, I get the message "Native serial port, can't obtain info"

The arduino-cli version is 0.25.1 Commit: 436f0bb9 Date: 2022-07-25T15:01:55Z
The arduino-cli.yaml has the line "additional_urls: [https://www.pjrc.com/teensy/td_156/package_teensy_index.json]"
If I run arduino-cli board list, I get
Error initializing instance: Error loading hardware platform: discovery teensy:teensy-discovery not installed
Port Protocol Type Board Name FQBN Core
/dev/ttyAMA0 serial Serial Port Unknown

If I run "arduino-cli core install teensy:avr", i get
Error initializing instance: Error loading hardware platform: discovery teensy:teensy-discovery not installed
Error initializing instance: Error loading hardware platform: discovery teensy:teensy-discovery not installed
Platform teensy:avr@1.57.0 already installed

What step I'm missing?

[PaulStoffregen]

Maybe run commands to uninstall and reinstall the teensy board package? Maybe if run with the verbose flag, info about *why* arduino_cli didn't install teensy-discovery might be shown?

[ptalbot]

Maybe run commands to uninstall and reinstall the teensy board package? Maybe if run with the verbose flag, info about *why* arduino_cli didn't install teensy-discovery might be shown?

Thanks Paul for the suggestion as it allowed me to go much further but not to the final destination. Now the arduino-cli board list looks good

pi@openplotter:~/Arduino/Nmea_Controller $ arduino-cli board list
Port Protocol Type Board Name FQBN Core
/dev/ttyAMA0 serial Serial Port Unknown
usb1/1-1/1-1.4 teensy Teensy Ports Teensy 4.1 teensy:avr:teensy41 teensy:avr

However, the upload gets stuck on the programm mode button that I have pressed without success. The log is:

arduino-cli upload -b teensy:avr:teensy41 -p usb1/1-1/1-1.4 -v
"/home/pi/.arduino15/packages/teensy/tools/teensy-tools/1.57.0/teensy_post_compile" "-file=Nmea_Controller.ino" "-path=/tmp/arduino-sketch-CB8FFE89FA18F3BAFD03419971A48628" "-tools=/home/pi/.arduino15/packages/teensy/tools/teensy-tools/1.57.0" "-board=TEENSY41" -reboot "-port=usb1/1-1/1-1.4" "-portlabel={serial.port.label}" "-portprotocol={serial.port.protocol}"
Teensy did not respond to a USB-based request to enter program mode.
Please press the PROGRAM MODE BUTTON on your Teensy to upload your sketch.
Error during Upload: Failed uploading: uploading error: exit status 1

I have tried with the Arduino IDE 1.8.19 and I get an error during the uploading

[ptalbot]

Also, the Teensy Loader version 1.57
I have tried a different USB port with the same problem

[PaulStoffregen]

Did you install the udev rules? It's required on all Linux systems.

[ptalbot]

Did you install the udev rules? It's required on all Linux systems.

Thanks a lot Paul as this is exactly what was missing. I took the udev info at https://www.pjrc.com/teensy/loader_linux.html

[PaulStoffregen]

Maybe the Linux version of teensy_post_compile should check if the udev rules are installed?

Until recently, this program was only delivered by the Teensyduino installer which checks if the udev file is present and matches the latest version (at the time the installer was built) with a dedicated GUI to advise installing the udev rules. But now with the Arduino CLI / IDE2 package, there's another path to installing teensy_post_compile and teensy_reboot, where you don't get any guidance to install the udev rules.

Or maybe we should request Arduino to provide a way to install this as part of installing the packages?