Arduino/Teensy. Would like ability to retrieve server client that hasn't sent data.

Status
Not open for further replies.
Title says it all. Current Arduino API will not return a socket (wrapped in a Ethernet Client class) unless it's received data on that socket. I'm trying to implement an FTP server with passive mode on a Teensy 3.5. The passive data socket will never receive data from the FTP client, only send. Further, the client will hang if the data socket is not closed by the server.

So first attempts at just using server.write do send the data, but since I have no way to close the socket, the FTP client hangs.

The base Arduino Ethernet library does make EthernetClient::status() and EthernetClass::_server_port public members. So I believe I could iterate through all sockets in userspace and find my open server socket to solve this problem. However the teensy version EthernetClass::socketStatus() is private and EthernetServer::server_port is commented that it will become private soon. So I can't iterate through sockets in userspace to find my established server connection.

For now, I may try to just use server.writes and the public EthernetServer::server_port to iterate through and close all connections that match the server port.

Open to other suggestions as well.

(Looks at code again...)

OK, EthernetClient::status() provides public access to EthernetClass::socketStatus(). So i was able to get this implemented using non-documented API calls.

Code:
for (int sock = 0; sock < MAX_SOCK_NUM; sock++) {
        EthernetClient data(sock);
        if (dataServer.server_port[sock] == FTP_DATA_PORT_PASV) {
          uint8_t s = data.status();
          if (s == 0x17 || s == 0x1C) {
            //assume its the first established one we find with the server port number
              //data = myclient;
              pasvDataSocket = sock;
              return true;
            }
        }
        
      }

So this is working for me, i iterate through all the sockets till i find one that matches my saved port number and shows as TCP established in status. But since this is undocumented API the Teensy Ethernet library does not match the Arduino Ethernet library (where the port numbers are saved) so this isn't portable. Would be nice if we could establish a documented call to retrieve connected clients that haven't sent data.

Ok, now on to figure out why every TCP packet is taking 200ms to ACK and severely slowing down the file transfer....
 
Ok, now on to figure out why every TCP packet is taking 200ms to ACK and severely slowing down the file transfer....

And in case anyone else finds this:

EthernetServer.cpp line 9

Code:
sockindex = Ethernet.socketBegin(SnMR::TCP, _port);

to

Code:
sockindex = Ethernet.socketBegin(SnMR::TCP | SnMR::ND, _port);

Sped that up. Though it sends 5 acks for every frame now. Guess that's the tradeoff.
 
Let's talk about an extension for the EthernetServer class.

It's looking likely that Arduino will soon accept Teensy's Ethernet lib (fully replacing theirs, which only supports W5100) and I'll take over most Ethernet lib maintenance. So now's a good time to think about how to permanently fix this for all of Arduino....

Also, please use 1.41-beta3, if you're not already. Several important updates to Ethernet have happened since 1.40 was released.
 
Odd that a sketch actually locks up on a Teensy 3.5 with a too high (not *that* high though) clock on a peripheral...

Indeed that would be odd, if the lockup were only due to overclocking the SPI.

But apparent lockups are surprisingly common for all sorts of other reasons. To illustrate, let me describe for you a problem we heard repeatedly in the early days of Teensy 3.0 and 3.1. People would report that Teensy wasn't retaining their code after power cycling. They would upload from Arduino and everything would work fine. Then when they power cycled the board, it would be unresponsive. The behavior so strongly implanted the idea that Teensy had not retained their code that it was nearly impossible to get people who experienced this problem to do almost anything to actually investigate the problem. They were absolutely convinced Teensy had not retained the program they uploaded, and virtually nothing could shake they belief once it had taken hold.

Of course Teensy was retaining their code. Some cases were a stray "while (!Serial) ;" left over in setup(), but the more common cause was their code or a library they used initializing some other chip, usually a motion sensor. Those chips had a brief startup time before they can respond. When uploading, the chip has already been powered on for quite some time. But on a cold boot, it would tend to be in that startup phase, unable to communicate. Many of those motion sensors start in a low power mode, and begin running when sent a command to start up. Most of the programs people used were libraries developed for regular Arduino boards which have a lengthy bootloader step before they start running. They would sent the hardware init command, and never do any checking whether the part actually responded, never retry or wait for the hardware to actually be ready. Then the rest of their program would be unresponsive, because it depended on getting good data from the sensor. Teensy was indeed running their program, but the program was never designed to properly handle the fault cases.

My guess is this lockup is very likely something similar, where the SPI running fast is giving wrong results, and the rest of the code isn't designed to deal with whatever isn't happening as expected. The challenging part is diagnosing the problem. Like with the rapid startup issue, there's a very natural human tendency to quickly form conclusions that fit a theory based on the parts that are known, without considering the larger system effects (especially undefined software behavior) which aren't as well known. Once those theories are formed, it's incredibly difficult to focus effort on investigating the unknowns...

In any case, the "Teensy didn't retain my code after power cycle" is now a thing of the past, due to a 400 ms delay that was added to the startup code, before C++ constructors and setup() are run. Simple as that sounds, it actually took a couple years to fully understand the issue and add that delay. That's how tricky these sorts of "lockup" problems can be!
 
About the EthernetServer class limitation, detecting clients which do not transmit data upon connecting, let's call this problem "EthernetServer can't do FTP".

Even if the Arduino devs allow me to take over maintenance of the Ethernet lib (seems likely sometime in 2018), this sort of API addition to a long established part of Arduino is going to require some effort to convince people of its value. Talk about protocol level stuff isn't going to make a strong impression on most people. But saying "can't do FTP" will.

My hope is we can add a basic FTP server example to the library. Does anyone have any code they'd like to contribute?
 
Last edited:
My hope is we can add a basic FTP server example to the library. Does anyone have any code they'd like to contribute?

I started with https://github.com/gallegojm/Arduino-Ftp-Server

And hacked it to work. This was mostly changing over to teensy SD calls and by iterating through all ports at a low level an checking for ones with an "established" status that match the server port number to catch new clients that have not sent any data (as per the FTP standard). See my first post. It's messy, but working for me, right now.
 
I've added EthernetServer.connected() to Teensy's Ethernet library.

https://github.com/PaulStoffregen/Ethernet/commit/7969cfba7d1fe23101c1762dae11436fedc17aac

Any chance I could talk you into testing the latest library with your FTP server?

The Arduino devs have agreed to accept Teensy's Ethernet for version 2.0.0 of the official Arduino Ethernet library, so this improvement (and so many others) will soon become available for all Arduino boards.
 
Another update. This is changing to server.accept(), where the main difference is it will return the client only once. Some changes may be needed within your library to keep track of the client until it disconnects.

Cristian Maglie (the lead Arduino dev) wanted this change. Long-term, I believe it'll be very good for Arduino. This behavior is much closer to how traditional socket API networking operates.
 
Back from the dead update.

Arduino 1.8.9 on a Teensy 3.5.
That Ardunio FTP server project (https://github.com/gallegojm/Arduino-Ftp-Server ) has had a lot of work done on it. It must use the new Ethernet additions to "make FTP work" without hacking, because, well, it's working. Haven't read it's code yet.

EDIT Read code! Look! ---> client = ftpServer.accept(); Yay you are the best!


It uses Arduino-FatLib now which is a wrapper for the different sd file libraries (SDfat, FatFs, SPIFFS) into a common structure. It needed some hacking to work on teensy (namely just changing a parent class to SdFatSdioEX from SdFat in ExtSdFat.h, should probably add a Teensy define for that someday)

BUT transfers are still slow, <1MB/s. With or without that EthernetServer.cpp hack. Using a W5200. Any idea why it's slow? SD card tests at 15MB/s, so I bet it's still related to Ethernet itself.
 
Last edited:
Status
Not open for further replies.
Back
Top