Teensyduino 1.59 Beta #2

luni · Apr 20, 2023

Did put a PR in to Paul and reported to @brtaylor.

I saw your PR. Wouldn't this PR eliminate Pauls macros completely? Probably a risky thing? Maybe just converting the macros to functions which shouldn't clash because the STL lives in the std:: namespace? Anyway, as long as undefining gets rid of the macros I'm fine with working around in my libraries.

Since the libs weren't mine I added ifdef/undef to the top of the sketch before those libs.

This helps only if the library doesn't have other c/cpp files which could run into the same issue. Took me some time to figure out which of the TimerTool / TeensyStep files needed to be adapted to make sure (hopefully) to eliminate the issue.

mjs513 · Apr 20, 2023

luni said:
I saw your PR. Wouldn't this PR eliminate Pauls macros completely? Probably a risky thing? Maybe just converting the macros to functions which shouldn't clash because the STL lives in the std:: namespace? Anyway, as long as undefining gets rid of the macros I'm fine with working around in my libraries.

This helps only if the library doesn't have other c/cpp files which could run into the same issue. Took me some time to figure out which of the TimerTool / TeensyStep files needed to be adapted to make sure (hopefully) to eliminate the issue.

Could be - just don't know enough about the risks but now that you mention it might very well - probably should just kill the PR.

Could very well be. The Eigen library is all headers so it seems to work. The UBX lib is only one .cpp file.

KurtE · Apr 20, 2023

PaulStoffregen said:
As mentioned on this thread, I have a preliminary 1.58.1 release to fix the "invalid library: no header files found" problem with latest IDE.

Code:

https://www.pjrc.com/teensy/td_158-1/package_teensy_index.json

Any chance I could talk you into giving 1.58.1 a quick try? Planing to go live sometime tomorrow.

Will also roll it into 1.59-beta3 and pull in those display library updates (and maybe other stuff), near the end of this week.

Worked as far as I could tell.

B3 - for me no hurry, just mentioning them now if there were some pressing stuff to get B3 out

PaulStoffregen · Apr 20, 2023

Thanks for testing!

I've updated the main package index, so all Arduino IDE 2.x.x users will see the 1.58.1 update.

jmarsh · Apr 20, 2023

I've found a bug in extmem_calloc and I'm not 100% sure of the best way to fix it:

Demo sketch:

Code:

#define SIZE_TO_TEST (4096)

extern "C" uint8_t external_psram_size;

void setup() {
  Serial.begin(9600);

  if (external_psram_size > 0) {
      // allocate all but 8KB
      void *p = extmem_malloc((1<<20)*external_psram_size - 8192);
      // attempt to allocate more (which will fail and end up being allocated from regular heap)
      uint32_t *m = (uint32_t*)extmem_calloc(sizeof(uint32_t), SIZE_TO_TEST);
      if (m) {
        size_t i;
        for (i=0; i < SIZE_TO_TEST; i++) {
          if (m[i] != 0) {
            Serial.println("calloc did not clear memory!");
            break;
          }
        }
        if (i == SIZE_TO_TEST) Serial.println("All OK");
        extmem_free(m);
      }
      else Serial.println("extmem_calloc failed");
      extmem_free(p);
  }
  else Serial.println("No extmem found to test");
}

void loop() {
}

The bug is that calloc (in this case extmem_calloc) is meant to zero out the returned memory. extmem_calloc is implemented as a wrapper of extmem_malloc - this is usually ok (if extmem is present!) because extmem_smalloc_pool has the do_zero flag set, which zeroes allocations before they are returned. But if extmem_malloc fails to get the memory from the extmem pool, it uses a regular malloc call to get it which does not zero out the returned memory.
If no external memory is present, extmem_calloc will always returned non-zeroed memory.

A quick fix would be to rewrite extmem_calloc to be the same as extmem_malloc except call calloc instead of malloc if the extmem allocation fails. But I think additionally, it should use sm_calloc_pool (instead of sm_malloc_pool) and the do_zero flag in the pool should not be set; this would improve performance by not unnecessarily clearing memory for regular allocations. A one-time clearing of _extram_start to _extram_end can be done when sm_set_pool is called in startup.c to make sure BSS variables are zeroed.

(Additionally the IS_EXTMEM macro in extmem.c has a typo - the parameter is named addr but the macro instead uses ptr, which works anyway because it's the local variable name.)

KurtE · Apr 20, 2023

Could it just be done like:

Code:

void *extmem_calloc(size_t nmemb, size_t size)
{
	size *= nmemb;
	void *ptr;
#ifdef HAS_EXTRAM
	ptr = sm_malloc_pool(&extmem_smalloc_pool, size);
	if (ptr) return ptr;
#endif
	ptr = malloc(size);
	if (ptr) memset(ptr, 0, size);
	return ptr;
}

jmarsh · Apr 20, 2023

It could, but why not just use calloc instead of malloc+memset?

Code:

void *extmem_calloc(size_t nmemb, size_t size)
{
#ifdef HAS_EXTRAM
	void *ptr = sm_malloc_pool(&extmem_smalloc_pool, nmemb*size);
	if (ptr) return ptr;
#endif
	return calloc(nmemb, size);
}

KurtE · Apr 20, 2023

Good question? You could always run both ways like 100 calls and see if there is any differences in speed...

In the mean time, you could probably setup a quick and dirty workaround. Not sure where you are calling this, as I don't see any in TGX nor ILI9341_t4,
I do see calls to the extmem_malloc()...
But if it were called and you are still using your macro(TGX_IS_EXTMEM)
if it is not EXTMEM you could memset it at that point.

jmarsh · Apr 20, 2023

It's not related to any library, it's just a generic issue with extmem_calloc (as the sketch shows).

I doubt there would be any difference in speed but in the case where someone compiles for T4.0 or the micromod, extmem_calloc would end up being one direct branch instruction to calloc.

KurtE · Apr 20, 2023

jmarsh said:
It's not related to any library, it's just a generic issue with extmem_calloc (as the sketch shows).

I doubt there would be any difference in speed but in the case where someone compiles for T4.0 or the micromod, extmem_calloc would end up being one direct branch instruction to calloc.

Sounds good. Maybe do a PR to Paul?

KurtE · Apr 20, 2023

Just an FYI - Suppose you have IDE2 running with Teensy install 0.59.2 and did a build or two and then decide to try out the 1.58.1 build and install it...
Then decide to then reinstall 0.59.2 you might end up with a setup the the builds will fail.

I have run into this twice now (actually 3 times if you include this time when I was doing the steps to create an Issue up on Arduino) https://github.com/arduino/arduino-ide/issues/2028

What happened is I still had Teensy.exe running. The install of the different version tried to remove the directories like TeensyTools and failed (no warning messages) and then when I went back to that version, it noticed that directory already existed and assumed everything was there. When actually only Teensy.exe was there....

defragster · Apr 20, 2023

KurtE said:
Just an FYI - Suppose you have IDE2 running with Teensy install 0.59.2 and did a build or two and then decide to try out the 1.58.1 build and install it...
...
What happened is I still had Teensy.exe running. ...

Ick - if Teensy_ports Sermon is open - does it do the same on that?

KurtE · Apr 20, 2023

I am not sure... Someone should probably try...

defragster · Apr 20, 2023

KurtE said:
I am not sure... Someone should probably try...

funny what you did there doing what I trying ... but you made the issue

Okay went back to 0.59.2 from 1.58.1 with Teensy_Ports SerMon open and the only folder in ...Local\Arduino15\packages\teensy\hardware\avr is the 0.59.2 so it managed to remove the 1.58.1.

>> SerMon stayed open and all seems to be okay?

Though that may be an issue with the 'newness' ordering in ..\tools\teensy-tools and teensy-monitor, etc as the JSON calls them out as needed for use/replacement?

Given it wasn't 'closed' if an update/delete is ever tried it would fail the remove. Though those things are in 'tools' and there seem to be multiple ver # folders under the tools - unlike the hardware\avr that only holds a single folder for the CORES.

defragster · Apr 21, 2023

KurtE said:
I am not sure... Someone should probably try...

After doing the p#64 swtich back to 0.59.2 I had not done build/upload.

Did a Build and it failed last night and tool [IIRC teensy_post_compile.exe] was missing from this directory:

Code:

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\0.59.2

[.]          [..]         teensy.exe
               1 File(s)      4,483,816 bytes

I copied from the IDE 1.8.19 tools folder and then I could get an upload.

Code:

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\0.59.2

[.]                       [..]                      mktinyfat.exe             precompile_helper.exe
stdout_redirect.exe       teensy.exe                teensy_gateway.exe        teensy_ports.exe
teensy_post_compile.exe   teensy_reboot.exe         teensy_restart.exe        teensy_secure.exe
teensy_serialmon.exe      teensy_size.exe
              12 File(s)      6,181,048 bytes

KurtE · Apr 21, 2023

Thanks @defragster

You might want to update my Arduino IDE Issue I mentioned in the previous post.

Also I am about 99% sure that it would not mater, but in the last run I know I had put a copy of platform.local.txt in the directory for TyCommander with the delegate.

Probably should try it with plain Vanila to make sure that it still happens....

defragster · Apr 21, 2023

Of course the platform.local.txt typically in use here too - though after I went to 1.58.1 I did not bother when had SerMon open.

The way some tools update with JSON links to older versions? Not sure what to expect bouncing around the versions - but for sure there are missing files in p#65 dir.

And there are two tools dirs - none from 1.58.? - and when looked both only had that single .exe in??? <EDIT>: And that EXE was CLOSED during the Brd Mgr Update.

Code:

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools

[.]      [..]     [B][COLOR="#FF0000"][0.59.1] [0.59.2][/COLOR][/B]

defragster · Apr 21, 2023

Did install of the updated 1.58.1 and dirs went from first below to the second. Where the 0.59.1 has more than one EXE because of manual copy. Then ALL removed after update to 1.58.1 and they are in that directory/folder - that was not removed? Though the single EXE 0.59.1 remains.

Code:

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools

[.]      [..]     [0.59.1] [0.59.2]
               0 File(s)              0 bytes

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\0.59.1

[.]          [..]         teensy.exe
               1 File(s)      4,483,816 bytes

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\0.59.2

[.]                       [..]                      mktinyfat.exe             precompile_helper.exe
stdout_redirect.exe       teensy.exe                teensy_gateway.exe        teensy_ports.exe
teensy_post_compile.exe   teensy_reboot.exe         teensy_restart.exe        teensy_secure.exe
teensy_serialmon.exe      teensy_size.exe
              12 File(s)      6,181,048 bytes

Code:

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools

[.]      [..]     [0.59.1] [0.59.2] [1.58.0]
               0 File(s)              0 bytes

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\0.59.1

[.]          [..]         teensy.exe
               1 File(s)      4,483,816 bytes

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\0.59.2

[.]  [..]
               0 File(s)              0 bytes

 Directory of C:\Users\Tim\AppData\Local\Arduino15\packages\teensy\tools\teensy-tools\1.58.0

[.]                       [..]                      mktinyfat.exe             precompile_helper.exe
stdout_redirect.exe       T4Security.jar            teensy.exe                teensy_gateway.exe
teensy_ports.exe          teensy_post_compile.exe   teensy_reboot.exe         teensy_restart.exe
teensy_secure.exe         teensy_serialmon.exe      teensy_size.exe
              13 File(s)      6,191,304 bytes

The 1.58.0 files in place before the 0.59.2 files were deleted ...

PaulStoffregen · Apr 21, 2023

KurtE said:
I have run into this twice now (actually 3 times if you include this time when I was doing the steps to create an Issue up on Arduino) https://github.com/arduino/arduino-ide/issues/2028

What happened is I still had Teensy.exe running. The install of the different version tried to remove the directories like TeensyTools and failed (no warning messages) and then when I went back to that version, it noticed that directory already existed and assumed everything was there. When actually only Teensy.exe was there....

I replied on issue #2028.

defragster · Apr 21, 2023

Just added new comment to 2028 as well - going 1.58.1 to 0.59.2 failed with 'dir exists' noted as error - but dir is empty - though under "computer control" 'In use' somehow cannot be deleted.
It was made empty on install of 1.58.1 - but not removed noted in p#68 - perhaps because the open CmdLine window was "in" that folder ... but stepping '..' did not allow its removal in dos box, or with Explorer.

mjs513 · Apr 22, 2023

Got a bit curious about what would happen if going between versions other than what was mentioned. So here goes.

Actually had to run the comparisons twice. On the first set of install downgrades/upgrades experienced compile errors (only IDE Open) but after accidentally closing the IDE and reopening installs seemed to working between upgrades/downgrades (with and without TeensyLoader open)

Run #1:

Code:

1,58,1 to 1.58.0:  (INSTALL)
recompile test sketch: ok
----------------------------

1.58.0 to 0.59.1:  (INSTALL)
recompile test sketch: ok
----------------------------

0.59.1 to 0.59.2:  (INSTALL)
recompile test sketch: ok
----------------------------

0.59.2 to 0.59.1:  (INSTALL)
recompile test sketch: ok
----------------------------

0.59.1 to 1.58.1   (UPDATE)
ERROR ON COMPILE
exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist

Compilation error: exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist
----------------------------

1,58,1 to 1.58.0  (INSTALL)
Tool teensy:teensy-tools@1.58.0 already installed
Tool teensy:teensy-compile@11.3.1 already installed
Tool teensy:teensy-discovery@1.58.0 already installed
Tool teensy:teensy-monitor@1.58.0 already installed
Downloading packages
teensy:avr@1.58.0......
INSTALL stops and does not complete.  To fix.  Closed IDE REMOVE 1.58.1 and reinstalled 1.58.0/
----------------------------

1,58.0 to 1.58.1 (UPDATE)
ERROR ON COMPILE
exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist

Compilation error: exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist
----------------------------

1,58,1 to 0.59.2 (INSTALL)
recompile test sketch: ok
----------------------------

0.59.2 to 1.58.0 (INSTALL) :
Error on COMPILE
exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist

Compilation error: exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist

1,58.0 to 0.59.2 (INSTALL):
recompile test sketch: ok
----------------------------


0.59.2 to 1.58.0 (INSTALL) :
rror on COMPILE
exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist

Compilation error: exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist
----------------------------

1.58.0 to 1.58.1 (UPDATE):
Error on COMPILE
exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist

Compilation error: exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist

----------------------------

REMOVED TEENSY REINSTALLED 0.59.2
recompile test sketch: ok
----------------------------


0.59.2 to 1.56.1 (INSTALL):
recompile test sketch: ok
Interesting when I checked TeensyLoader about it showed 1.57.1??
----------------------------

1,56,1 	back to 0.59.2 (INSTALL)
recompile test sketch: ok
----------------------------

0.59.2 to 1.56.1 (INSTALL):
recompile test sketch: ok

1,57.2 to 1,58.1 (UPDATE):
ERROR on COMPILE
exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist

Compilation error: exec: "C:\\Users\\Merli\\AppData\\Local\\Arduino15\\packages\\teensy\\tools\\teensy-tools\\1.58.0/precompile_helper": file does not exist


update to 1.58.0
recompile test sketch: ok

Run 2

Code:

--------------------Closed IDE 2.1.0 by accident and then reopened------------------------------
1.58/0 to 0.59.2 (INSTALL)
recompile test sketch: ok
----------------------------

0.59.2 to 1.58.0 (INSTALL):
recompile test sketch: ok
----------------------------


0.59.2 to 1.58.1 (UPDATE):  
recompile test sketch: ok
----------------------------

1.58.1 to 0.59.2 (INSTALL)
recompile test sketch: ok
----------------------------


0.59.2 to 1.56.1 (INSTALL)
recompile test sketch: ok
Interesting when I checked TeensyLoader about it showed 1.57.1??
----------------------------

1.56.1 to 1.57.1 (INSTALL)
recompile test sketch: ok
----------------------------

1.57.1 to 1.58.1 (UPDATE)
recompile test sketch: ok
----------------------------

1.58.1 to 0.59.2 (INSTALL) with TeensyLoader 1.58 open
recompile test sketch: ok

----------------------------

0.59.2 to 1.58.0 (UPDATE) with TeensyLoader 1.58 open
recompile test sketch: ok

PaulStoffregen · Apr 22, 2023

luni said:
I saw your PR. Wouldn't this PR eliminate Pauls macros completely? Probably a risky thing?

We're still early enough in 1.59 beta to risk this. These Arduino marcos have been causing trouble for C++ code for years. If removing them turns out to break important Arduino stuff, I'll (probably) put them back in.

KurtE · Apr 26, 2023

@Paul

Short Version:

Code:

void setup() {
  while(!Serial);
}
void loop() {
  delay(1000);
  Serial.println(Serial.baud(), DEC);
}

If I run this and use the COM29 under Teensy ports on either IDE1 or 2,
This prints out as 0's for the baud.

However if I choose the Com29 under Serial ports, it prints out as 115200
(Or whatever I change the baud to in the Serial Monitor window).

Ditto for TyCommander.

I found this when working on the USB to Serial type sketches, that it was seeing the 0 and assigning that to the USB Host Serial object (or Hardware Serial object in other main _teensy example USB sketch). The sketch now detects the 0 and does not use it...

Should the Teensy ports support setting the Baud rate?

PaulStoffregen · Apr 27, 2023

KurtE said:
If I run this and use the COM29 under Teensy ports on either IDE1 or 2,
This prints out as 0's for the baud.

Ran it here. I'm also seeing 0 on Windows for both IDE 1.8.19 and 2.1.0. On MacOS and Linux it prints 9600 in both IDE versions.

KurtE · Apr 27, 2023

Not sure if you control it in the pluggable monitor stuff? Or if the Arduino side?
As I mentioned, I switched to the Arduino serial monitor, which has the Baud drop down list, and the sketch saw it when I changed the setting.

Teensyduino 1.59 Beta #2

Well-known member

Senior Member+

Senior Member+

Well-known member

Well-known member

Senior Member+

Well-known member

Senior Member+

Well-known member

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Well-known member

Senior Member+

Senior Member+

Well-known member

Senior Member+

Well-known member

Senior Member+