Teensyduino 1.54 Released

Status
Not open for further replies.
Yes.


The size print looks like this:
Code:
RAM:   [===       ]  26.6% (used 139616 bytes from 524288 bytes)
Flash: [          ]   0.9% (used 75736 bytes from 8126464 bytes)
Building .pio\build\teensy41\firmware.hex
..which is not as good as the one Teensy has for T4/4.1/MM
which looks like this (more or less (I use a slightly modified version))

Code:
(FB) Memory Usage on Teensy 4.1:
  FLASH: code:26884, data:3784, headers:8240   free for files:8087556
   RAM1: code:24264, variables:17088   free for local variables:474432
   RAM2: variables:12384  free for malloc/new:511904

Hi @Frank B! The latest upstream version of the Teensy platform now also uses the "teensy_size" tool to report additional info about memory usage. It'd be great if you could retest your project using the following configuration:

[env:teensy41]
platform = https://github.com/platformio/platform-teensy.git
framework = arduino
board = teensy41
 
Hi,

Its allready for a long time possible to get more info on the allocated memory during compilation,
just add this to platformio.ini
Code:
build_flags = -Wl,--print-memory-usage
Gives this output
Code:
Memory region         Used Size  Region Size  %age Used
            ITCM:      151036 B       512 KB     28.81%
            DTCM:      123584 B       512 KB     23.57%
             RAM:       15424 B       512 KB      2.94%
           FLASH:        325 KB      7936 KB      4.10%
            ERAM:     6432512 B        16 MB     38.34%
 
Its allready for a long time possible to get more info on the allocated memory during compilation,
just add this to platformio.ini

That is better, but still not nearly as good as the new size report.

In particular, ITCM and DTCM are also both reported as their maximum 512K size, rather than their actual size within RAM1.

Flash memory is also reported as a single number, which gives no visibility into code vs data vs header/overhead.
 
Hi Paul,

Unfortunately it seems the new size report did not make it into the 1.54 that is currently stable on PIO. Just a few days ago that was announced but I dont get a detailed report. Maybe the next time.
This report at least helps me to understand where my large chuncks of data are and how much space I still have.
As suggested by valeros I also tried to use the "upstream" version but ... did not get the new reports also. Probably this demands a rebuild of PIO again, removing cached versions etc.

cheers
Cor
EDIT: After removing cached versions of teensyduino in platformio;
Code:
Memory region         Used Size  Region Size  %age Used
            ITCM:      151036 B       512 KB     28.81%
            DTCM:      123584 B       512 KB     23.57%
             RAM:       15424 B       512 KB      2.94%
           FLASH:        325 KB      7936 KB      4.10%
            ERAM:     6432512 B        16 MB     38.34%
Checking size .pio/build/teensy41/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=====     ]  52.4% (used 274608 bytes from 524288 bytes)
Flash: [          ]   3.7% (used 299336 bytes from 8126464 bytes)
teensy_size .pio/build/teensy41/firmware.elf
teensy_size: Memory Usage on Teensy 4.1:
teensy_size:   FLASH: code:179364, data:144836, headers:8596   free for files:7793668
teensy_size:    RAM1: variables:123584, code:151032, padding:12808   free for local variables:236864
teensy_size:    RAM2: variables:15424  free for malloc/new:508864
teensy_size:  EXTRAM: variables:6432512
 
I am on linux but I think its the same on windows.
You need to go to the .platformio directory (hidden)
Then go to packages
and remove all framework-arduinoteensy subdirectories

Than rebuild your code and hopefully that works. I use the output to understand "globally" how my data and code are distributed but the "project inspector" inside platformIO allows
you to check in full detail which variable/function is stored where etc etc.

cheers
Cor
EDIT: The inspector allows you to check things in detail
Selection_999(007).png
 
Hi @Frank B! The latest upstream version of the Teensy platform now also uses the "teensy_size" tool to report additional info about memory usage. It'd be great if you could retest your project using the following configuration:

[env:teensy41]
platform = https://github.com/platformio/platform-teensy.git
framework = arduino
board = teensy41

If I do this, i get :
Code:
Platform Manager: Installing git+https://github.com/platformio/platform-teensy.git
[COLOR=#ff0000]Error: Traceback (most recent call last):
  File "c:\users\frank\.platformio\penv\lib\site-packages\platformio\commands\run\processor.py", line 70, in process
    p = PlatformFactory.new(self.options["platform"])
  File "c:\users\frank\.platformio\penv\lib\site-packages\platformio\platform\factory.py", line 54, in new
    raise UnknownPlatform(pkg_or_spec)
platformio.platform.exception.UnknownPlatform: Unknown development platform 'https://github.com/platformio/platform-teensy.git'

[...] etc[/COLOR]

So.. back to platform = teensy

@Corbee: Hm, does not help... i try to find out how to remove & reinstall teensy platform completely. Interestingly, the size tool is there - it just gets not used.
 
Uninstalling Teensy Platform via "PIO Home" and re-install does not work for me, too. Still:
Code:
[...]
Linking .pio\build\teensy41\firmware.elf
Checking size .pio\build\teensy41\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [===       ]  26.6% (used 139616 bytes from 524288 bytes)
Flash: [          ]   0.9% (used 75736 bytes from 8126464 bytes)
Building .pio\build\teensy41\firmware.hex
Configuring upload protocol...
[...]

So... I have no Idea how to get it working.

(p.s corbee yes, i have have used --print-memory-usage before. it's great.)
 
My platformio.ini looks like this and works fine.
Code:
[env:teensy41]
platform = https://github.com/platformio/platform-teensy.git
board = teensy41
framework = arduino
build_flags = -Wl,--print-memory-usage
 
I upgraded to 1.54 (clean- uninstalled Arduino, then reinstalled), along with VisualTeensy 1.0.0

I can compile, but get these multiple definition errors with FNET. I'm using the one in Teensyduino, v0.1.3 or 4.7.0 as defined in fnet.h.

Code:
.vsteensy/build/project17.elf
Memory region         Used Size  Region Size  %age Used
            ITCM:      173084 B       512 KB     33.01%
            DTCM:       63232 B       512 KB     12.06%
             RAM:       12416 B       512 KB      2.37%
           FLASH:        406 KB      7936 KB      5.12%
            ERAM:          0 GB        16 MB      0.00

.vsteensy/build/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.o: In function `fnet_mimxrt_serial_init':
C:\Users\win10\github\project17/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.cpp:56: multiple definition of `fnet_mimxrt_serial_init'
.vsteensy/build/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.o:C:\Users\win10\github\project17/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.cpp:56: first defined here
c:/progra~2/arduino/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld.exe: Disabling relaxation: it will not work with multiple definitions
.vsteensy/build/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.o: In function `fnet_mimxrt_serial_init':
C:\Users\win10\github\project17/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.cpp:56: multiple definition of `fnet_mimxrt_serial_putchar'
.vsteensy/build/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.o:C:\Users\win10\github\project17/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.cpp:56: first defined here
.vsteensy/build/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.o: In function `fnet_mimxrt_serial_init':
C:\Users\win10\github\project17/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.cpp:56: multiple definition of `fnet_mimxrt_serial_getchar'
.vsteensy/build/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.o:C:\Users\win10\github\project17/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.cpp:56: first defined here
.vsteensy/build/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.o: In function `fnet_mimxrt_serial_init':
C:\Users\win10\github\project17/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.cpp:56: multiple definition of `Serials'
.vsteensy/build/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.o:C:\Users\win10\github\project17/lib/FNET/src/port/cpu/mimxrt//fnet_mimxrt_serial.cpp:56: first defined here
collect2.exe: error: ld returned 1 exit status
%
make: *** [makefile:237: .vsteensy/build/project17.elf] Error 1
The terminal process "C:\windows\System32\WindowsPowerShell\v1.0\powershell.exe -Command C:/Users/win10/github/VisualTeensy_v1_0_0_0/make.exe all -j -Otarget" terminated with exit code: 1.
 
Does this also happen if you use the Arduino IDE?
If it turns out to be a VisualTeensy problem it would be good to file an issue with an example on the VisualTeensy repo.
 
Does this also happen if you use the Arduino IDE?
If it turns out to be a VisualTeensy problem it would be good to file an issue with an example on the VisualTeensy repo.

Thanks luni, it looks to be a VisualTeensy or library issue on my end, I'll start a new thread.
 
NativeEthernet & FNET TCP Server Fixes

I've been looking into this NativeEthernet bug. Indeed the crash is happening deep within FNET. It's going to take some serious work to unravel to figure out what the actual cause really is.

Just to be realistic, this is very unlikely to get fixed in 1.54. So many other things are updated & improved, and it's been just over 1 year since 1.53 and Arduino 1.8.15 is now 7 weeks old. We really need to wrap up 1.54 now.

But I definitely will be diving into this problem in the coming weeks. It will get fixed, though a problem this complex isn't the sort of thing that (typically) gets solved quickly.

Here are trimmed copies of the code to reproduce the problem. I sped it up quite a lot, so it usually crashes in less than 1 minute.

I've finally dedicated some time to hopefully fixing this and I may have done it though some more testing should be done to catch any edge cases that could be happening and verifying that no data is being lost like it used to be.

For NativeEthernet I've already made a commit because I've previously tested this fix when I first looked into this, it stops NativeEthernet from actually running out of sockets and completely stopping when FNETs TCP code doesn't behave like you would expect it too.
https://github.com/vjmuzik/NativeEthernet

As for FNET, I've narrowed it down to where the data was getting thrown out though I haven't figured out why exactly since as far as WireShark can tell the packet that was getting thrown out was the same as previous ones that had no issues. So I've commented out the lines where it would close the socket that was making NativeEthernet have issues and as far as I can tell with Apache Benchmark there doesn't seem to be any missed packets or dropped connections like there used to be. If this fix works for everyone without causing any side effects I will go ahead and merge it.
https://github.com/vjmuzik/FNET/tree/TCPListenPatch-1

Code for testing:
Code:
#include "NativeEthernet.h"

uint8_t mac[6];
void teensyMAC(uint8_t *mac) {
    for(uint8_t by=0; by<2; by++) mac[by]=(HW_OCOTP_MAC1 >> ((1-by)*8)) & 0xFF;
    for(uint8_t by=0; by<4; by++) mac[by+2]=(HW_OCOTP_MAC0 >> ((3-by)*8)) & 0xFF;
    Serial.printf("MAC: %02x:%02x:%02x:%02x:%02x:%02x\n", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
}

//EthernetServer server(443, true); //Uncomment for TLS
EthernetServer server(80);

void setup() {;
  // put your setup code here, to run once:
  teensyMAC(mac);
  Ethernet.begin(mac);
  MDNS.begin("Teensy41", 2); //.local Domain name and number of services
//  MDNS.setServiceName("Teensy41_Service_Name"); //Uncomment to change service name
//  MDNS.addService("_https._tcp", 443); //Uncomment for TLS
  MDNS.addService("_http._tcp", 80);
  server.begin();
  Serial.print("IP  address: ");
  Serial.println(Ethernet.localIP());
  Serial.send_now();
}

void loop() {
  // listen for incoming clients
  EthernetClient client = server.available();
  if (client) {
    Serial.println("new client");
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        Serial.write(c);
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connection: close");  // the connection will be closed after completion of the response
//          client.println("Refresh: 1");  // refresh the page automatically every 5 sec
          client.println();
          client.println("<!DOCTYPE HTML>");
          client.println("<html>");
          // output the value of each analog input pin
          for (int analogChannel = 0; analogChannel < 6; analogChannel++) {
            int sensorReading = analogRead(analogChannel);
            client.print("analog input ");
            client.print(analogChannel);
            client.print(" is ");
            client.print(sensorReading);
            client.println("<br />");
          }
          client.println("</html>");
          client.close(); //If "Connection: close" make sure to close after printing and before stop to avoid harsh reset
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        } else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
    Serial.println("client disconnected");
  }
}

Apache Benchmark Command
Code:
ab -k -c 10 -n 10000 http://teensy41.local/
 
@vjmuzik, thanks for working on this.

This is my first attempt to use NativeEthernet, but I got your fix, built and installed your test program, and then ran the Apache Benchmark. It was taking a long time, so just to experiment I reduced concurrency from 10 to 1, and number of iterations from 10000 to 10. I'm not sure what I'm looking for, but here is typical output of "ab" for that test.

Do you expect to have some "Failed Requests"?

C:\AppInstalls\Apache>ab -k -c 1 -n 10 http://teensy41.local/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking teensy41.local (be patient).....done


Server Software:
Server Hostname: teensy41.local
Server Port: 80

Document Path: /
Document Length: 199 bytes

Concurrency Level: 1
Time taken for tests: 2.119 seconds
Complete requests: 10
Failed requests: 8
(Connect: 0, Receive: 0, Length: 8, Exceptions: 0)
Keep-Alive requests: 0
Total transferred: 2628 bytes
HTML transferred: 1998 bytes
Requests per second: 4.72 [#/sec] (mean)
Time per request: 211.912 [ms] (mean)
Time per request: 211.912 [ms] (mean, across all concurrent requests)
Transfer rate: 1.21 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 3 2.1 3 8
Processing: 200 208 4.4 209 213
Waiting: 1 4 1.9 3 7
Total: 203 212 4.4 214 215

Percentage of the requests served within a certain time (ms)
50% 214
66% 214
75% 214
80% 214
90% 215
95% 215
98% 215
99% 215
100% 215 (longest request)
 
Hopefully there shouldn’t be any failed requests anymore, 10000 requests took less than a minute for me.
 
I misread your post and didn't realize I had to update both FNET and NativeEthernet. I've now downloaded your latest code for both from Github, and I'm getting the error in building your test sketch. Should I not have downloaded the latest FNET and NativeEthernet from Github? I'm using Arduino 1.8.15 and TD 1.54 on Windows 10.


Code:
C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\NativeEthernet\src\NativeEthernet.cpp:39:31: error: conflicting declaration 'void** EthernetClass::socket_ptr'
 fnet_socket_t* EthernetClass::socket_ptr;
                               ^
In file included from C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\NativeEthernet\src\NativeEthernet.cpp:23:0:
C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\NativeEthernet\src\NativeEthernet.h:91:36: note: previous declaration as 'void* volatile* EthernetClass::socket_ptr'
     static volatile fnet_socket_t* socket_ptr;
                                    ^
C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\NativeEthernet\src\NativeEthernet.cpp:39:31: warning: declaration of 'void* volatile* EthernetClass::socket_ptr' outside of class is not definition [-fpermissive]
 fnet_socket_t* EthernetClass::socket_ptr;
                               ^
 
Last edited:
For NativeEthernet I've already made a commit because I've previously tested this fix when I first looked into this, it stops NativeEthernet from actually running out of sockets and completely stopping when FNETs TCP code doesn't behave like you would expect it too.
https://github.com/vjmuzik/NativeEthernet

I've been having intermittent problems with Native Ethernet which were characterised by it sending corrupted frames when busy. I am using it with MQTT, and also noted a possibly related long delay to make a connection and frequent connection breakages.
Disconnecting the ethernet cable would sometimes provoke a crash.

I hadn't got a repeatable bit of test code to submit for this, but as it's recently got worse, I checked for updates before investigating and noted a well-timed release with several ethernet fixes in 1.54 so have installed it (also arduino 1.8.15).

First indications are that a big improvement - or at least, a big change - is there. The delay on connecting has gone and I am now performing longer tests.

Is this an area that's been improved and where I can expect to see fixes ? If there's a thread on noted ethernet bugs I'd appreciate a pointer.
 
I'm getting same issue with latest versions in Github. The 1.54 Teensyduino release libs are good.

warning:
Code:
lib/FNET/src/stack//fnet_tcp.c: In function '_fnet_tcp_initial_seq_number_update':
lib/FNET/src/stack//fnet_tcp.c:450:70: warning: suggest parentheses around '+' in operand of '&' [-Wparentheses]
     _fnet_tcp_initial_seq_number += FNET_TCP_INITIAL_SEQ_NUMBER_STEP + fnet_rand() & 0xFF;

errors:
Code:
lib/NativeEthernet/src//NativeEthernet.cpp:39:31: error: conflicting declaration 'void** EthernetClass::socket_ptr'
 fnet_socket_t* EthernetClass::socket_ptr;
                               ^
In file included from lib/NativeEthernet/src//NativeEthernet.cpp:23:0:
lib/NativeEthernet/src//NativeEthernet.h:91:36: note: previous declaration as 'void* volatile* EthernetClass::socket_ptr'
     static volatile fnet_socket_t* socket_ptr;
                                    ^
lib/NativeEthernet/src//NativeEthernet.cpp:39:31: warning: declaration of 'void* volatile* EthernetClass::socket_ptr' outside of class is not definition [-fpermissive]   
 fnet_socket_t* EthernetClass::socket_ptr;
                               ^
make: *** [makefile:213: .vsteensy/build/lib/NativeEthernet/src//NativeEthernet.cpp.o] Error 1
make: *** Waiting for unfinished jobs....
 
Sorry for the error I mistakenly didn't copy over that file with the changes, it's there now though. The warning in FNET is also now fixed, if I wasn't so bad at version control I would've noticed earlier that my local branch was behind and I would've seen that sooner.
 
vjmuzik, I got your latest FNET and NativeEthernet, and your test program now builds and runs. When I run Apache Benchmark, there are now no errors, but it is much slower for me than what you said (10000 requests in < 1 minute). With concurrency = 1, each request takes about 200 ms (~2.1 s for 10 requests). If I increase concurrency, it gets slower. I have the T4.1 plugged into my wireless router, but my connection from PC is wireless. Maybe this is my network, as opposed to having anything to do with FNET/NativeEthernet?
 
Certainly your network setup, I test with a USB to Ethernet connection to my laptop and the latency is generally 2-4ms.
 
I switched to ethernet cable, still going through router, and I'm getting the same (slow) results, but I'll wait for others to do some testing, as I really know very little about Ethernet or networks.
 
Status
Not open for further replies.
Back
Top