Teensy Qt

[Theremingenieur]

Un grand merci de l'est vers le nord!

I didn't want to recompile it if it isn't absolutely necessary, but rather deploy it "as is". The only thing I'd like to change is the "Teensy 3.1" string, but I don't yet know if it comes from TyQt or from the Teensy itself. I looked up in the usb_desc.h file of the teensy core, but could not find it there...

 

[Koromix]

De rien

The name comes from TyQt, look here: https://github.com/Koromix/ty/blob/maintenance/src/board_teensy.c#L111

This information is not exposed directly by the Teensy. The model is exposed only when the bootloader is running, as a single numerical value in the usage field of the primary HID collection. It's the "usage" field in the model struct I linked above.

The good news is, both TyQt and tyc share a library (libty.dylib) to interface with the Teensies and this library is easy to compile and does not need Qt. I think the build system will error out if you don't have Qt, but I can change it to instead disable the GUI when building without it. You can then reuse the binaries I provide, and replace libty.dylib with your own.

I don't use Xcode and I don't know a single thing about it. clang and gcc are supported though. I provide Qt Creator project files and a CMake-based build system, both work on OS X.

 

[Samuel]

This is absolutely fantastic! Especially with the serial monitor. Without TyQT, the last thing you really needed the Arduino IDE for was the serial monitor - with TyQT, that isn't the case anymore. This means it's feasible (and even easy!) to work Arduino-less. Thank you so much!

 

[Theremingenieur]

De rien

The name comes from TyQt, look here: https://github.com/Koromix/ty/blob/maintenance/src/board_teensy.c#L111

This information is not exposed directly by the Teensy. The model is exposed only when the bootloader is running, as a single numerical value in the usage field of the primary HID collection. It's the "usage" field in the model struct I linked above.

The good news is, both TyQt and tyc share a library (libty.dylib) to interface with the Teensies and this library is easy to compile and does not need Qt. I think the build system will error out if you don't have Qt, but I can change it to instead disable the GUI when building without it. You can then reuse the binaries I provide, and replace libty.dylib with your own.

I don't use Xcode and I don't know a single thing about it. clang and gcc are supported though. I provide Qt Creator project files and a CMake-based build system, both work on OS X.

Before setting up everything to compile/build a new version of libty.dylib, I'll give the Hex Editor a try to simply modify the strings in the compiled version...

 

[Koromix]

Update: TyQt 0.7.0 is now out, you can find the last release in the original post.

TyQt 0.7.0 is almost ready, and I wanted to release a pre-version for those interested:
- Windows: https://www.dropbox.com/s/8mbwbyc4r2cp8ru/TyQt-0.6.3-278-gf419560-win32.zip?dl=0
- OSX: https://www.dropbox.com/s/0516oj8m9z5tly9/TyQt-0.6.3-278-gf419560-osx.dmg?dl=0

The biggest improvement is the ability to work with multiple boards at once. Select multiple boards (using Ctrl), click on "File > Upload New Firmware" and pick all the firmwares you want. The first one compatible with each board will be used.

Here is an animated GIF to illustrate this: http://imgur.com/O7T6eqJ

You can also reboot/reset multiple boards at once the same way. ATM, this functionality is not exposed on the command-line, which is what I want to change before releasing TyQt 0.7.0.

Other changes (unordered):
- Visual warning on task errors
- Board tags are now "<serial>@<location>"
- JSON output for `tyc list` using `tyc list -Ojson`
- Select board using device path in tyc/tyqt (e.g. "tyc upload --board @COM1")
- Faster firmware uploads
- Static binaries (no dependency on libty.dll)
- Replace "Upload" tab with "Settings" tab
- Show last uploaded firmware in board list
- Support El Capitan (osx)
- Add tyqtc to pilot TyQt (win32)
- Use short COM port names when possible (win32)
- Make README.txt/LICENSE.txt notepad-friendly (win32)
- Drop missing boards after a longer delay
- Add keyboard shortcut to clear monitor (Ctrl + Alt + X)

Main changes in release TyQt-0.6.3-155-g5be58ce:
- Various improvements to monitor performance to avoid buffer overruns
- Fix freeze of serial read in some cases on Win32
- Increase size of monitor scrollback in TyQt to 400000 lines
- Disable word-wrap in monitor text widget
- Avoid modal errors dialogs on startup for non-fatal errors
- Remove annoying LICENSE prompt from OSX bundles

Main changes in release TyQt-0.6.3-167-g1097f80:
- Read serial data from a dedicated thread to avoid stalls and overruns
- Allow selection of missing boards in TyQt
- Better handling of device close on Windows XP

Main changes in release TyQt-0.6.3-199-g81d2743:
- Improved command-line for TyQt (not complete yet)
- Fix tyqtc dropping output/error data
- Make TyQt autostart opt-in (--autostart)
- Add Arduino integration dialog

Main changes in release TyQt-0.6.3-222-gdef7db2:
- Board status icons replace the generic board icon
- Resizable board list using a QSplitter
- Hide harmless spurious I/O errors (generated when the Teensy is busy writing the previous binary chunk) when uploading
- Add Upload toolbar submenu with 'Upload all'
- Fix crash when selected firmwares are invalid in TyQt

Main changes in release TyQt-0.6.3-245-gc9d5dbd:
- Add board attach/detach function to better cooperate with other Teensy-related softwares
- Minor UI/GUI changes and improvements
- Fix crash with tyc in quiet mode

Main changes in release TyQt-0.6.3-250-g0a71b13:
- Keep board list width fixed when resizing main window
- Potential fix for deadlock issue on Windows

Main changes in release TyQt-0.6.3-257-g76eca29:
- Support integration to legacy Arduino (1.0.x)
- Respect Arduino upload verbosity setting

Main changes in release TyQt-0.6.3-278-gf419560:
- Change board tag syntax to "<serial>-<family>"
- Extend board selection syntax to "[<serial>][-<family>][@<location>]"
- Allow board renaming/aliasing in TyQt
- Save board settings and alias
- Darker icons (main and status icons) and other minor UI changes
- Fix monitor send button (it never worked ^_^)

The code has changed quite a bit since TyQt 0.6.4 so this pre-release is probably not as stable. The project is now distributed under the terms of the MIT license.

 

[Wayne]

I just gave this a go and it works great.

Thanks

 

[Koromix]

I've just uploaded a new build that works better on Windows (Windows 7/8/XP are slow to detect devices changes when you trigger parallel resets). It also fixes a crash when selecting invalid firmware files.

 

[defragster]

Downloaded and installed - Win 10 - Minimal use - No Problems!

Simple uploads and restarts to one T_3.2.

Another T_3.2 and T_3.1 running existing sketch - reset just fine. Is it expected that the 'Teensy' identifier is not set to actual type until I hit RESET? On first power and repower they are just 'Teensy'.

Found ports for a pair of T_LC's - one came up 'Teensy' - then went to "LC" on reset.

The second LC was button reprogrammed as it was going 'missing' - even then it starts and goes missing?

I reverted to the prior (September) version and it shows all 6 devices.

Returning to the latest download on start up I get this:


Note - the prior TYQT also only showed 5 units including BOTH LC's - I had to repower the unshown unit - A T_3.2 - then it showed all 6.

It seems like there is some problem on seeing more than 5 in some fashion. This behavior repeated going:: Newest > old > newest > old > newest TYQT. Except again the missing unit is now a T_3.2 and on the Newest is shows when repowered like the old version.

NOTE: Two Teensy's show they are COM5! Which is what WINDOWS 10 is doing ???? Here is the system WMIC report: View attachment tyqt.txt

 

[Koromix]

Thank you for the tests

The modal message box for permission errors is a regression, and I just fixed it locally. The COM thing however... Looks like Microsoft hasn't fixed Windows' 10 USB/serial stack all that well. I don't see many solutions except limiting parallel uploads/reboots/resets to 2 or 3 on Windows. And maybe detect when Windows f*cks up to inform the user.

I tested on Windows 7 on my machine, and Windows 10 in a VM. I had some odd behavior on Windows 10 but the VM USB layer has all kind of weird bugs so I haven't looked deeply into it. Unfortunately I only own 4 Teensies.

I will look into it.

 

[Koromix]

Another T_3.2 and T_3.1 running existing sketch - reset just fine. Is it expected that the 'Teensy' identifier is not set to actual type until I hit RESET? On first power and repower they are just 'Teensy'.

This is normal, and it was already the case with previous releases. The model can only be identified when the bootloader is running. After that TyQt remembers it.

 

[defragster]

After that TyQt remembers it.

I figured it was normal - was that way before. Even after a device goes missing and returns though - with Serial # shown though it shows 'Teensy'.

I take it you don't have any static storage in the registry or a config.file? The 'clear serial on reset' is not sticky across TYQT restarts. Also I think I noted before it would be nice to have a 'friendly name' associated with a serial # - but without static storage that would be tough - and even then it might have value limited to the task at hand - though the name of the last sketch uploaded would be cool but not always the same machine or run through TYQT even if you saw the name - so many features for you to add

When I reboot if I see different behavior I'll post the notes.

BTW: If the printed USB log from Windows was useful:
<open as CMD window (run as administrator)>
type/run>POWERSHELL
in powershell>Get-WMIObject Win32_SerialPort
then>Exit
To save a disk file named "usbports.tx" in POWERSHELL>Get-WMIObject Win32_SerialPort > usbports.txt

 

[Koromix]

You're absolutely right, there is no persistence at all But this is something I've wanted for some time, so I'll come to it. And it will allow TyQt to remember the serial number <=> model mapping. In fact, I still have a "prefs.patch" file around but it was not finished and it is 6 months old at this point, so it doesn't work with the current code.

The master branch of ty was stuck in a broken state for months, and I have finally restored some order to the source code. With that done, I can go back to regular releases with 1 or 2 features each time.

I'm not sure in which order I will implement the rest, but this is what I'm thinking of:
- send file / dump serial to file (0.7.1)
- persistent preferences (0.7.2)
- board aliases (0.7.3)
- ignore board (0.7.4)

BTW: If the printed USB log from Windows was useful:
<open as CMD window (run as administrator)>
type/run>POWERSHELL
in powershell>Get-WMIObject Win32_SerialPort
then>Exit
To save a disk file named "usbports.tx" in POWERSHELL>Get-WMIObject Win32_SerialPort > usbports.txt

Yes it is, thank you. I don't know that much about Windows, I just hack around until it works. Well, educated hacking, but still.

 

[defragster]

@Koromix: I like the Ctrl+Alt+X - thanks nice addition! Good Work.

If you want to see what I am doing with it today check out : Data-dropouts-in-serial-transfer-over-USB
> Note: dropping back to the awful IDE SerMon I see it running twice as fast taking data from the Teensy - I know you sped it up but sketch here may be interesting.

> QUESTION: I am seeing that sketch FREEZE on MAX USB transfers (same has happened in the IDE SerMon). I'm wondering if you could add a 'DROP and RECONNECT USB'? [Ctrl+shift+X ?] In this case the USB stalls as noted - but if I change between TYQT and IDE SerMon it wakes the transfer up? Or close and reconnect SerMon but doing that with TYQT takes longer. This would be a momentary 'Ignore Board' in some fashion perhaps. I'm not sure if this behavior points to a Windows or Teensy loss of I/O - but the restart resumes it.

<edit: Closing TYQT and restarting it does NOT restart the sketch like SerMon does?>

 

[defragster]

I posted my updated USB sketch: Data-dropouts-in-serial-transfer-over-USB?

Putting in my qBlink() properly I see the Teensy is perfectly alive suggesting the PC it the one getting overwhelmed - even though I am throttling the Teensy output to a mere 4Mbit/sec where the Teensy never blocks emptying the buffer.

Somehow the IDE SerMon close and restart picks the port back up better. Having TYQT clear and close and reopen the port would be helpful. As noted in above link the 66byte write takes as little at 130 usecs (continuous for 40,000 writes) to the SerMon and at or over 700 usecs on to TYQT so there is room for improvement.

Note: I restarted TYQT and it shows my T_3.2 running - but I had to power cycle the T_3.1 to get it to start showing USB data?

Also noteworthy - TYQT will generally pickup the connection after a button press (on any sketch) - where my Win10 with IDE 1.6.6 will rarely if ever reconnect and the SerMon must be closed and restarted, SerMon does restart with UPLOAD handshake process though - it would be cool if TYQT did too.

 

[Koromix]

Thanks for the report. I've just uploaded a new release that improves on the monitor performance and fixes some things, you can find the info in the pre-release post.

Somehow the IDE SerMon close and restart picks the port back up better. Having TYQT clear and close and reopen the port would be helpful. As noted in above link the 66byte write takes as little at 130 usecs (continuous for 40,000 writes) to the SerMon and at or over 700 usecs on to TYQT so there is room for improvement.

I've improved the code, is it better now? It's not as fast as SerMon, the IDE uses a dedicated thread to read from the board and I don't. With these fixes, I think it's good enough so I don't think the extra complexity is warranted. Of course the command-line version (tyc) has no problem keeping up... but throw in a single-thread GUI event loop and "slow" text widgets, and things get more complicated.

> QUESTION: I am seeing that sketch FREEZE on MAX USB transfers (same has happened in the IDE SerMon). I'm wondering if you could add a 'DROP and RECONNECT USB'? [Ctrl+shift+X ?] In this case the USB stalls as noted - but if I change between TYQT and IDE SerMon it wakes the transfer up? Or close and reconnect SerMon but doing that with TYQT takes longer.

The last release (0.6.3-155) contains a fix for a bug in the Win32 serial read code, where the code would stop issuing read requests in some non-rare cases. Maybe this is what was happening?

<edit: Closing TYQT and restarting it does NOT restart the sketch like SerMon does?>

I know the Arduino boards do this, but it must not be working over here, because I've never seen that happen with a Teensy. But anyway, is that really a feature? I can add that as a setting when TyQt gets persistent settings, but I don't like that by default.

 

[defragster]

Downloaded - will let you know. I never saw or used TYC that way, will try it!

Quote Originally Posted by defragster View Post

<edit: Closing TYQT and restarting it does NOT restart the sketch like SerMon does?>

I apparently I misquoted myself - I wasn't looking for auto restart - what I meant to say was

<edit: Closing TYQT and restarting it does NOT restart the sketch SERIAL I/O TRANSFER like SerMon does?>

NOTE: OLD TYC - it fails after a short time - not tried new - but on exit and restart of TYC - it shows 'Connection ready' - but doesn't pick up the transfer in progress - will try new ASAP.

Of course doing that restart a couple times with IDE SerMon often gets JAVA out of memory error in IDE.

 

[defragster]

AWESOME WORK - YOU FIXED TYQT REALLY WELL!!! It runs faster than the fastest solution I found with at least one run catching all 40,000 strings of 70 characters in 3.858 seconds!

The last TYQT version would not make 2 full passes against my USB sketch before it hung - it has run over 79 passes and stayed connected and is in fact running faster than IDE SerMon! It keeps running perfectly! I have restarted and uploaded new code and the whole thing is working and running as it should!

Also works better than TYC! TYC still will not pick up in progress serial stream - I have to restart the Teensy for TYC to being showing data.

This snapshot shows it after completing 4 runs of 40,000 prints of 70 character strings. Two cases - with and without send_now that flushes USB buffers and slows things down rather than multitasking the hardware. STILL RUNNING!!!!!

The faster 'NO SEND NOW' case looks like 5.8 Mbps! fastest I saw before was 4.2 mbps direct to file in a dos box.

The other send_now case is also running almost 2 seconds faster where the code has added 112 usecs of deliberate wait to the execution loop in order to keep the data from backing up - as noted the send_now case forces the Teensy to empty buffers and breaks up the continuous optimal background data flow. The code detects this 'blocking' by watching an elapsedMicroseconds value and increases the wait outside the print loop to keep it down so there is no blocking within the loop.

Here is my sketch as it stands: View attachment USB_Throttle2.ino

40000 loops with ___#runs:4 ___

__throttle [ NO SEND NOW ] usecs =100
__1000 xfer ms time=103__1000 xfer Wait Loops ms=18370__Test Time ms=3853

__throttle [ with SEND NOW ] usecs =212
__1000 xfer ms time=215__1000 xfer Wait Loops ms=47263__Test Time ms=7980______

 

[defragster]

Update - on the NO SEND NOW case it can run faster - I had a lower threshold wait of 100 usecs (line 13 : "#define THROTTLEINIT 50"). When I drop that it works up as needed and it is working reliably with a throttle wait of only 80 usecs against TYQT. This is 6.9 Mbps! it is running faster and 100% reliably!

40000 loops with ___#runs:13 ___

__throttle [ NO SEND NOW ] usecs =80
__1000 xfer ms time=84__1000 xfer Wait Loops ms=12933__Test Time ms=3105

__throttle [ with SEND NOW ] usecs =230
__1000 xfer ms time=235__1000 xfer Wait Loops ms=52724__Test Time ms=8666______

<Edit> - I was overshooting on throttle increment - it runs reliably with 58 usec wait with TYQT:

__throttle [ NO SEND NOW ] usecs =58
__1000 xfer ms time=62__1000 xfer Wait Loops ms=6951__Test Time ms=2319

This is 1,155,670 bytes per second - or 9.245364 mega bits per second

<Edit 2:> this is too fast - some 1224 strings are lost at this rate with GUI display - but it still runs! {this was at 58 usecs throttle, going to 60 usecs stops data loss)

<edit3:> Update running for 175 cycles my sketch was seeing TYQT slow down - I expect due to GUI and buffer management (it is getting HUGE piles of data with over 2.6 MB every testing cycle as fast as every 2.4 seconds) It has to dump and recycle the buffer space - task manager showed the app taking 385 MB - when I dumped the display buffer it dropped to 200 MB! it just did a natural dump and was under 120MB for a second! Another buffer dump manually went to under 20MB for the app - it is buffering a lot of data!

 

[defragster]

No Complaints on TYQT - it ran many iterations and then over 500 more in a single session! I dumped the buffer Ctrl+Alt+X and the RAM dropped to 20MB from over 200MB!

Then it did a couple iterations more and windows said it needed to DIE so it killed TYQT - then one more time after that. I just restarted TYQT and it picked up fine - quickly holding 200MB of buffered data again.

198 iterations ran on command line into a file that was 529MB. These runs with retries, because I have the throttle initialized purposefully low, to make it seek it have more overhead and only update throttle after pushing too much data too fast - so over 1.5 GB of USB data the last 4 hours?

I set throttle up from 50 usecs (that went to 60) to 90 usecs and that is working in 3.462 seconds per cycle with all 40,000 lines accounted for in one cycle and 77 full cycles completed without rebooting from the crashes. That gives TYQT the needed breathing room - a whole spare second!

Great tool Koromix that makes working with Teensy even better!

As I noted above the only good thing about SerMon is that it can be told by the IDE to drop the USB port to allow programming automatically (otherwise SerMon is much more troublesome and problematic), I don't know that TYQT can do this. But TYQT has UPLOAD and RESET buttons that allow fine grained individual control of any Teensy online with just a click when more than one is attached - nothing lese does that! And now it has a better faster more reliable serial monitor than IDE SerMon and Putty and "MegunoLink Pro"!

Koromix: Is there a reason the GUI TYQT is better than the command line TYC on my system?

 

[Koromix]

I've just uploaded a new build with faster and more reliable monitor, also available in the pre-release post.

Then it did a couple iterations more and windows said it needed to DIE so it killed TYQT - then one more time after that.

Yep, turns out the workaround I found to "accelerate" the main Qt event loop didn't work like I thought it did. And it lead to a stack overflow after a while, hence the crashes Well, I caved in yesterday and added a thread dedicated to reading serial. That's what this new pre-release is about.

As I noted above the only good thing about SerMon is that it can be told by the IDE to drop the USB port to allow programming automatically (otherwise SerMon is much more troublesome and problematic), I don't know that TYQT can do this.

With 0.7.0 I will release instructions to integrate it with the IDE and replace Teensy Loader. If all goes well, this integration will upload to the selected board in the IDE (in Serial mode), or provide a board selection dialog on first sketch upload for the other modes.

Also works better than TYC! TYC still will not pick up in progress serial stream - I have to restart the Teensy for TYC to being showing data.

I can't reproduce it. However, if your sktech is in the "pause phase", tyc will not show anything until it resumes. Also, tyc takes the first Teensy it finds unless you use "--board". Are you sure it's not monitoring another board? Also, by default "tyc monitor" exits when the board is removed, but you can use "--reconnect" to avoid that.

Koromix: Is there a reason the GUI TYQT is better than the command line TYC on my system?

Over here I don't find it is, on any platform. However, cmd.exe's scrolling is very slow. Maybe that's where the bottleneck is, try redirecting to a file.

 

[defragster]

Thanks for the feedback and effort. Downloading now . . . results in about an hour . . .

 

[defragster]

Downloaded and used to GOOD EFFECT!
<edit>The GUI scroll of data is much more linear - no apparent jumping as the input comes in at over 1MB/sec!

It is much more responsive! A couple hundred runs and Still does not fail like IDE SerMon does!

The RESET and BOOTLOADER upload per device in the GUI make it the perfect complement to the TEENSY!!!!! And now a super capable Serial Monitor!

Prior version was showing data loss - this is no longer the obvious case! You had much better buffer management - and now the dedicated thread keeps TYQT from missing data! 2.567 seconds to transfer 2.708894 MB is 1,055,276 MB/sec or 8.4 Mb/s with no apparent loss watching a couple runs. If you fixed a long term stack over flow that is way cool too.

Additionally another shorter burst example runs well against TYQT where before it was showing regular data loss! SerMon could handle that one

For TYC - I was likely using it wrong before and not being patient for output : With recent download though I get this error starting TYC:

tyc.exe - System Error
---------------------------
The program can't start because libwinpthread-1.dll is missing from your computer. Try reinstalling the program to fix this problem.

 

[Koromix]

Oups, my bad. My hack to force gcc to link libwinpthread statically has stopped working with gcc 5.3.0 and I didn't realize it. I've just fixed it and updated the pre-release for Windows.

Thank you a lot for your tests. With the monitor problems out of the way, I can probably release TyQt 0.7.0 soon.

 

[defragster]

- glad to help - TYQT makes Teensy better - which is saying something.

I can live with the IDE Dev tool so far (really must get VisStudio and VisualMicro set up - hope it plays well with TYQT) - but the IDE SerMon is very lacking and limiting given no other debug tool, and not having wired reset TYQT is awesome - as well as directed bootloading.

For instance I just installed the Teensy Logic Analyzer sketch - meaning a second Teensy online to use another one - having the IDE reprogram it would be a 5 minute ordeal. Which makes me think I should save that HEX file once I see it working and need to use it that way. However - TYQT will take up that USB_COM port and not let the logic analyzer software see it - OOOPS - have to run the L_Analyzer first to have it own the port until you have the 'TYQT IGNORE COM#' implemented.

 

[Koromix]

I've uploaded a new build with the poetic name TyQt-0.6.3-199-g81d2743. It features a new Arduino integration dialog, accessible under "Help > Integrate to Arduino". It will replace the Teensy Loader with TyQt, or do the reverse with a single button.



Some day I will release a stable version, before this project gets lost in feature creep land