Off the top of my head I only know of this made by manitou: https://github.com/manitou48/teensy4...t_tftpd_SPIFFS
Theoretically if there was an existing library that worked on the w5500 it should work with this, I just haven’t tested any myself. FNET does have one built in, but it’s a little bit lacking (no write functionality) so I haven’t spent any time getting it to work with a SD card.
Thanks for responding. I think I will start playing with the Arduino FTP server program I found and maybe even the ones I used with wifispi. There are some others that I was checking out but never finished working with. Finally have a good base to work with in the T4.1.Originally Posted by vjmuzik
Edit; With the T.36 I had both the FTP client and FTP server working with MSC and uSDFS. There is a lot more utility availble now with the T4.1.
Re: ftp server
Maybe something can be learned from the lwIP lib's ftp server. In 2016 with the T3.5/3.6 ethernet beta test we had a working ftp server that used lwIP and SdFat lib (v1). I couldn't port that to the T4.1 lwIP because SdFat-beta (v2) did not have the VFS support that the ftp lib needed. I did manage to get a limited ftp server to work on T4.1 that used lwIP and SD lib but only did get/put. In the git repository look at folders lwip_ftpd and lwip-ftpd-sd
from https://forum.pjrc.com/threads/60532...l=1#post237096
Code:microSD read times (seconds) (4.2MB file, 2KB reads) T4 SD lib T3.5 SdFatv1 T4.1 SdFatv2 T4.1 SPIFFS EFLASH tftp 4.76 2.2 2.2 s 2.3 s http 3.6 0.9 0.9 ftp 2.8 0.6 read() 2.76 0.26 0.19 s 0.2 s
@manitou - Again thanks for responding. You pointed me to a lot of info. I have worked through all of @vjmuzik's FNET examples and tftpd with success. Now to checkout lwIP.
So many things to explore and learn about. I am not very familiar with Ethernet yet and how it works
Thanks
vjmuzik,
I did some more testing with the network stack on the 4.1 tonight. There are two parts to our code, one is a UDP send and Receive, the other is a web page. If I do not initialize the web interface, the UDP stuff works with no issues and no lockups. If I just initialize the web server and client, i no longer get all the inbound UDP requests. Is there possibly an issue with more than one open port/stack at a time?
UPDATE: I did some more testing. If I comment out either the UDP or the WEB Server so only one is defined and being serviced, it works perfectly. This seems to point at a possible socket conflict.
-Steve
Last edited by shmorgan; 07-07-2020 at 03:33 AM.
I double and tripple checked, the stock ethernet library will not hang when the cable is NOT connected unless you are using DHCP. If you use a static IP address like we are, it never hangs during startup.
Not to my knowledge there isn’t any issue as long as no port addresses are shared, this might be related to a DNS issue with UDP that I haven’t pushed the fix for yet. You are correct if you use a static IP it doesn’t freeze, the original library would freeze while waiting to get an IP with DHCP and fail if it couldn’t obtain one. It’s an easy fix to change the functionality so it doesn’t lock up since it doesn’t really make sense to retain that functionality from the original. I’ll post the DNS fix shortly and work on DHCP a little bit later.
@manitou @vjmuzik - Have lwip_ftpd and lwip-ftpd-sd working with NativeEthernet. I originally had this Arduino FTP server working with the T3.6 and WIFISpi.
https://github.com/gallegojm/Arduino-Ftp-Server.
Since that time there has been several updates to this FTP server. The latest version uses:
https://github.com/gallegojm/Arduino-FatLib
I think this version of the library uses an older version of SdFat which does not seem compatible with SdFat 2.0. Anyway the library is setup to use SdFat, FatFS and SPIFFS on the Esp8266. It seems that only SdFat is actually implemented in the Arduino-FatLib library. SdFat is way out of my capabilities so I think I'll add @WMXZ's uSDFS to the FatLib library. Basically ExtFatFs using ExtSdFat as a template. Has anybody else played with this? Thoughts?
@All - SUCCESS! I now have the T4.1 version of Arduino-Ftp-Server giving me directory listings using FileZilla and gFTP. The library is using NativeEthernet, Arduino-Ftp-Server, Arduino-FatLib, Arduino-FatFS, uSDFS and MSC. The Arduino-FatFs library is a c++ wrapper for FatFs. Here is the link to the repositories of gallegojm:
https://github.com/gallegojm?tab=repositories
I think I have the basic structure of this figured out and it will take a lot of work to get everything working. When I have it all working I'll put it on Github. SPIFFS will be interesting with the recent work with the T4.1 and external memory. SdFat? It sounds like that may not be supported anymore. Is that right?
vjmuzik,
I wanted to check back and see if you might have any updates to the library that might fix the issue with Dual IP Stacks not playing well. And a fix for the Ethernet hanging during startup (with a static IP address) when a cable is not connected. I also have a program that duplicates the issue, but is a bit too large to upload here. Can you PM me an email address, I can send it your way.
-Steve
Last edited by shmorgan; 07-14-2020 at 01:08 AM.
I tried the NativeEthernet library in combination with the OSC library and the t4 audio shield. Works like a charm. Thank you for sharing this very user friendly library!
Attached is a working example for those who want to try out the same. I'm not a professional coder though, but it's useful for testing.
t41_osc_fft_test.ino
I was finally able to find a moment to create an example where packets seem to be dropped inbound to the library.
This example has both a Web Server (Port 80) and a UDP listener on port 21649. If I comment out the Web Server initialization code in setup, the code will respond to 100% of the poll packets sent from Packet Sender. Once the web server code is enabled, it drops a random number of UDP poll packets.
Having what I believe is the same problem. Working on the Arduino FTP server I am losing packets from server to client if the file size is > 512 bytes. If transfer from client to server there is no problem. File sizes are the same. If I delay the the transfer:It will transfer the file from server to client complete. "nb" always to the correct amount of bytes actually transferred no matter what size the the buffer is. So it seems as if the the teensy is loosing packets. Is there function or method that can wait for a transfer to complete before issuing another Transfer?Code:bool FtpServer::doRetrieve() { if( ! dataConnected()) { file.close(); return false; } uint32_t nb = file.read( buf, FTP_BUF_SIZE ); if( nb > 0 ) { data.write( buf, nb ); bytesTransfered += nb; delay(100); return true; } Serial.printf("nb = %d\n",nb); closeTransfer(); return false; }
Using Arduino 1.8.13 and TD 1.53
Edit: does not matter which ftp client I use. Filezilla, gFTP or ftp in linux.
Last edited by wwatson; 07-19-2020 at 02:57 AM. Reason: Typo again:)
@vjmusik - Dang me! I did not realize your update for NativeEthernet was 07/02/2020. I thought the update was included in the release of TD 1.53. Off buy one again which was 07/01/2002
Anyway, Everything seems to work with the ftpserver in both directions. At this point the optimal transfer buffer size seems to be 2k. As expected MSC does not perform as well with uSDFS file transfers as SDHC. I can get 2.2MHz download speeds from SDHC and ~235KBs upload speeds to SDHC. 167KBs upload speeds to thumb drives and 665KBs download speeds from flash drives.
FileZilla seems very hard to keep happy with transfers. gFTP is what I am using the most to test with. Ftpserver needs a lot of rework to make it compatible with the Teensy T4.1. But at this point it is working. Thanks for a great library. IT WORKS!
I couldn't reproduce your problem, though you did not specify the the UDP packet size and packet rate you were using. Also using a broadcast address
IPAddress remoteIP(255, 255, 255, 255);
is very anti-social. The common way to send a UDP reply is to use the IP and port from the sender.
Udp.beginPacket(Udp.remoteIP(), Udp.remotePort());
In my test i sent 48-byte UDP packet and had your sketch just send the same packet back using the construct above. My sender would wait for a reply or timeout for each packet sent. I didn't lose any UDP packets even with web client queries. The UDP RTT was about 280 us, so packet rate was about 3500 pkts/sec
Last edited by manitou; 07-20-2020 at 06:02 PM.
manitou, Can I get you to upload your test sketch?
I know that the 255,255,255,255 is not ideal, but is required in our specific application/situation. It is duplicating the functionality of an existing device.
-Steve
Well, I did some more tests and had inconsistent results. It turned out I had two Udp.begin() in setup(). I fixed that and cloned in the latest FNET and NativeEthernet. Now UDP echo only works if web client statements are commented out in loop(). With web client in loop() enabled, the UDP does not respond and the T4.1 freezes (have to push program button to upload new sketch)
NativeEthernet Library Hangs during startup when no link/cable is detected.
I did want to share this quick fix if you are also having issues with the network library hanging during startup/begin when the Ethernet cable is not connected.
Here is the quick fix.
You will need to locate where the Arduino environment placed the NativeEthernet-master library. You should be able to google this as it varies depending on your environment.
under the src directory, locate the NativeEthernet.cpp file and comment out the following two lines in the void EthernetClass::begin(uint8_t *mac, IPAddress ip, IPAddress dns, IPAddress gateway, IPAddress subnet) function.
Before:
After:Code:while(!link_status){ }
Code://while(!link_status){ //}
With regard to the Issue of dropped UDP receive packets when the web server is enabled, I was able to dig about the library and may have found something to lead us to what is going on.
In the NativeEthernetUdp.cpp file, I added a bit of logging and see that the socket is indeed receiving the bytes every time, but is also getting an error on about half of the inbound packets. There is a call to fnet_error_get() that returns -6 about half the time, which causes this function to return 0 for byte count.
Here is the code I modified to print out some logging...
This is a log from an error packet:
This is a log from a good packet:Code:Recv1: Socket Index: 0 Remaining _remaining: 12 Recv2: Socket Index: 0 Remaining ret: 12 Error -6
You can see that in both cases, 12 byes were read in, Not sure on the -6 error code, it is defined as FNET_ERR_INVAL, An invalid argument was supplied.Code:Recv1: Socket Index: 0 Remaining _remaining: 12 Recv2: Socket Index: 0 Remaining ret: 12 UDP read 12 Packet Contents: SOMETESTTEXT
In all cases, I am sending the exact same poll packet to the test code.
Code:if ((_remaining = Ethernet.socketRecvAvailable(sockindex)) > 0) { //HACK - hand-parse the UDP packet using TCP recv method struct fnet_sockaddr from; fnet_size_t fromlen = sizeof(from); Serial.print("Recv1: "); Serial.send_now(); Serial.print("Socket Index: "); Serial.println(sockindex); Serial.print("Remaining _remaining: "); Serial.println(_remaining); Serial.send_now(); fnet_ssize_t ret = fnet_socket_recvfrom(Ethernet.socket_ptr[sockindex], Ethernet.socket_buf_receive[sockindex], Ethernet.socket_size, 0, &from, &fromlen); Ethernet.socket_buf_index[sockindex] = 0; Serial.print("Recv2: "); Serial.send_now(); Serial.print("Socket Index: "); Serial.println(sockindex); Serial.print("Remaining ret: "); Serial.println(ret); Serial.send_now(); //read 8 header bytes and get IP and port from it int8_t error_handler = fnet_error_get(); if(ret == -1){ Serial.print("RecvErr: "); Serial.send_now(); Serial.println(error_handler); Serial.print("Socket Index: "); Serial.println(sockindex); Serial.print("Remaining: "); Serial.println(_remaining); Serial.send_now(); return 0; } if(error_handler == -6){ Serial.print("Error -6 "); Serial.send_now(); return 0; } if (ret > 0) { _remoteIP = from.sa_data; _remotePort = FNET_HTONS(from.sa_port); } // _remaining = 0; return ret; } // There aren't any packets available return 0;
One other observation, even on a successful receive of the bytes, the error_handler is getting a -4 FNET_ERR_AGAIN. Just for fun, I commented out the sections in NativeEthernetUdp.cpp that checks for error_handler == -6 and I receive 100% of my packets. Not sure this is correct or will be stable. This may be an indication of an underlying issue with the inbound buffers.
vjmuzik
I have the same problem as shmorgan had in post #37. I duplicated the modification to WebServer Example he showed and had the same result. Paul released 1.53 on July 1 which is what I assume I am running. Your post #38 on July 2 says you fixed the problem. I am not familiar enough with installing libraries to get your fix.
Can you please tell me how to get your fix. I did download the zip file from github and install it in user/Documents/Arduino. But, that created other errors.
Thank you
Placing the unzipped library folder\files in the IDE's 'Sketchbook' folder should properly find and use it after an IDE restart?
Unsure of the default : Arduino\libraries ??? - as I'm using edited path to sketchbook here outside the Arduino normal paths
In the IDE hit :: 'Ctrl+,' to see the path it uses in the properties dialog.
Hey. First of all, thank you for the great work on this library.
In last few days I have been trying to port a webserver library I made ages ago to work with this new library. I got it working now, but it seemed a bit slow when sending data, even for smaller pages. So I have made some tests just using the base examples to try and reproduce the problem.
I created a function to create a buffer of a specific size with a constant value, then send it
Then called that function on a loop, so I could send any amounts of data with different buffer sizes.Code:void sendBuffer(uint8_t client, uint8_t value, uint16_t size) { byte send_buffer[size]; memset(send_buffer, value, size); clients[client].write(send_buffer, size); //clients[client].flush(); checkTimer(); // Prints the time elapsed since startTimer() was called }
I also tried using different values for Ethernet.setSocketSize() in the setup.Code:startTimer(); for (size_t j = 0; j < 100; j++) { Serial.print(j); Serial.print("\t"); sendBuffer(i, 'A'+(j%26), 256); }
The values below are the result of the checkTimer() function call, printed after each client.write();
In this specific case I was using an 8K buffer in setSocketSize() and was doing 256 bytes writes each time.
Looking only at this, seemed like something happend when the buffer fills up. Each write took less then 10uS for the first 8KB, then there was a massive delay. I tried with different sockets and write size combinations and this is consistent. This would happen only after I write more than the buffer size.
Initially I assumed it would be a buffering thing. Maybe I was calling the writes faster than the data was being sent, and something would be stuck. But then, I also tried sending bigger amounts of data (several MB). I was expecting it to maybe hang up a few times along the way while it was catching up. But in every single test I made, it always has that one single hick-up the first time the buffer fills up, then it's fast all the way to the end. I also tried using the flush() method, but that made things way slower.0 Timer:0(S) 0(mS) 8(uS)
1 Timer:0(S) 0(mS) 12(uS)
2 Timer:0(S) 0(mS) 16(uS)
3 Timer:0(S) 0(mS) 20(uS)
4 Timer:0(S) 0(mS) 25(uS)
5 Timer:0(S) 0(mS) 29(uS)
6 Timer:0(S) 0(mS) 34(uS)
7 Timer:0(S) 0(mS) 38(uS)
8 Timer:0(S) 0(mS) 42(uS)
9 Timer:0(S) 0(mS) 47(uS)
10 Timer:0(S) 0(mS) 51(uS)
11 Timer:0(S) 0(mS) 56(uS)
12 Timer:0(S) 0(mS) 60(uS)
13 Timer:0(S) 0(mS) 65(uS)
14 Timer:0(S) 0(mS) 69(uS)
15 Timer:0(S) 0(mS) 74(uS)
16 Timer:0(S) 0(mS) 78(uS)
17 Timer:0(S) 0(mS) 82(uS)
18 Timer:0(S) 0(mS) 87(uS)
19 Timer:0(S) 0(mS) 92(uS)
20 Timer:0(S) 0(mS) 96(uS)
21 Timer:0(S) 0(mS) 101(uS)
22 Timer:0(S) 0(mS) 105(uS)
23 Timer:0(S) 0(mS) 110(uS)
24 Timer:0(S) 0(mS) 114(uS)
25 Timer:0(S) 0(mS) 119(uS)
26 Timer:0(S) 0(mS) 123(uS)
27 Timer:0(S) 0(mS) 128(uS)
28 Timer:0(S) 0(mS) 132(uS)
29 Timer:0(S) 0(mS) 137(uS)
30 Timer:0(S) 0(mS) 142(uS)
31 Timer:0(S) 0(mS) 146(uS)
32 Timer:0(S) 40(mS) 40334(uS)
33 Timer:0(S) 81(mS) 81285(uS)
34 Timer:0(S) 81(mS) 81290(uS)
35 Timer:0(S) 81(mS) 81295(uS)
36 Timer:0(S) 81(mS) 81299(uS)
37 Timer:0(S) 81(mS) 81304(uS)
38 Timer:0(S) 81(mS) 81902(uS)
39 Timer:0(S) 81(mS) 81907(uS)
40 Timer:0(S) 81(mS) 81911(uS)
41 Timer:0(S) 81(mS) 81916(uS)
42 Timer:0(S) 81(mS) 81921(uS)
43 Timer:0(S) 81(mS) 81925(uS)
So I went a little deeper and used Wireshark to look at the packages being sent. And here the problem seems to be in the very first packages. We can see it sent the first frame with 256 bytes, which was my first buffer. Then the second package (the one in gray) had a payload of 1460, containing data from multiple writes(). This package was sent 40ms after the first one. Then the next package was similar, and it took another 41ms. But after these two, all the other packages took way less than a milisecond.
I tried many different combinations, and the behaviour is always the same. The first write() goes in the first package. Then the next few writes() are packed together in the second package, and its transmitted around 40ms after the first. Then the third is typically the same. And after this, all seems to stabilize and there are no more delays. Even if i only do 2 small writes(), in the Teensy prints it says they took like 10uS, but in Wireshark I see the the second package being sent 40ms after the first one. Could the library maybe be waiting for more data to fill a frame or something like that? And when it actually has enough data it does not realize it and only sends the frame after a timeout? Now I'm just guessing...
Let me know your thoughts, and if you'd like to make any other tests let me know.
defragster
The real question is the version in github referred to in post #38 different from what was included in the teensydino version 1.53? I am not seeing any difference in operation.
That post was responding to a noted difference with the github version: "Can you please tell me how to get your fix. I did download the zip file from github and install it in user/Documents/Arduino. But, that created other errors."
If there is an error then the new code is not likely in use as indicated by "not seeing any difference in operation".
The verbose console output will indicate the source of the library used to compile. If pointing to the TD 1.53 copy that explains unchanged behavior. If that 'libraries used' indicates the copy in the sketchbook from the github zip without errors pasting that from the console would show that.
Posting Permissions
Reply With Quote