QNEthernet and SimpleFTPServer modification

P

Paul L

Member
I modified the setup files for SimpleFTPServer to work with QNEthernet, and SdExFat/ExFile. However, when downloading files, I rarely got the whole file unless the file was very small.

My solution was to replace the call to file.write( buf, nb ) in bool FtpServer::doRetrieve() to file.writefully( buf, nb ).

Modified Command

I don't know if this is unique to QNEthernet, but it sure seems like the FTP server should have a more robust routine for ensuring data is sent. I generally prefer keeping libraries unmodified so I don't need keep track of my own mods, but it seems necessary in this case. Perhaps a TFTP server would take care of this issue, but I don't want the speed reduction of constant replies.

On a side note, when programming on platforms that don't have multithreading capability, I like having the option of using non-blocking functions. To that end, I'm going to modify both QNEthernet and SimpleFTPServer to accommodate this. My project requires that CAN data, and other I/O are monitored and updated without interruption.
 
Good thing is @shawn is active in improving QNEthernet library when it makes general improvements.

Showing your work might help with that.
 
Last edited:
The QNEthernet library can be used in a non-blocking way. Libraries or code that use it need to be written in a way that uses this. Is it possible the SimpleFTPServer library just isn’t written in a non-blocking style?

Note: writeFully() isn’t a non-blocking function.
 
Last edited:
One other note is that TCP data does not get sent immediately unless you call flush() on the connection. If you don’t call flush(), the data is buffered until either there’s enough to send or a timeout of about 250ms occurs.
 
Note three: you must check return values from write() calls (and from all networking calls, for that matter). The writeFully() call hides the fact that not all bytes necessarily get written when write() is called.
 
Sounds good.

I was able to get the SimpleFTPServer working well with QNEthernet and meets my needs. The following is just to document what I did in case someone else wants to try it.

For some reason uploads are limited to approximately 10MB/s, and downloads 8MB/s. Maybe due to the old ethernet cable I'm using. I tried different size buffers to no avail. If I didn't call flush() after write(), the download is limited to approx 4MB/s.
Edit: 100Mbit/sec is roughly 10MB/s as @jmarsh points out below.

I did have to make a number of changes to SimpleFTPServer to get it to work as I wanted. I should note that SimpleFTPServer does not block as I thought, when implemented as-is, or with the new changes I've made.

Here are the changes to FtpServer.h at the end of class FtpServer:

Code: 
public:
void    setStorageManager(SdExFat* sd_ptr);

private:
//FtpServer was using "extern SdExFat STORAGE_MANAGER", and caused redefinition issues;
SdExFat* STORAGE_MANAGER; //all functions in FtpServer.cpp changed to pointer notation

int16_t nb_sent{0};  //number of bytes read from file that will be sent;
int16_t bytes_written{0}; //number of bytes actually transferred over ethernet
uint8_t* buf_ptr{nullptr}; //to keep track of buffer position


Changes to FtpServer.cpp:

Code: 
//
  //  RETR - Retrieve
  //
        bytes_written = 0; //added
        nb_sent = 0; //added

Code: 
bool FtpServer::doRetrieve()
{
 
  if( ! dataConnected())
  {
    file.close();
    return false;
  }
  if(bytes_written == nb_sent || nb_sent == 0)
  {
  
    nb_sent = file.read( buf, FTP_BUF_SIZE );
    buf_ptr = buf;
    bytes_written = 0;
  }
  if( nb_sent > 0 )
  {



    uint16_t bytes_sent = data.write( buf_ptr, nb_sent -  bytes_written);
    data.flush();
    bytes_written  +=  bytes_sent;
    buf_ptr += bytes_sent;
    bytesTransfered += bytes_written;

    DEBUG_PRINT(F("NB --> "));
    DEBUG_PRINTLN(nb_sent);
  
      if (FtpServer::_transferCallback) {
          FtpServer::_transferCallback(FTP_DOWNLOAD, getFileName(&file).c_str(), bytesTransfered);
      }

// RoSchmi
#if STORAGE_TYPE != STORAGE_SEEED_SD
    return true;
#endif
  }
  closeTransfer();
  return false;
}
 
Last edited:
It's a 100megabit ethernet port so with 8 bits per byte plus overheads you're not going to see much faster than 10MB/s.
 
You must log in or register to reply here.
Back
Top