T4.1 Ethernet issue

Frankthetech

Frankthetech

Well-known member
I'm working on a T4.1 with the ethernet kit and ran into a issue sending data to the ethernet port, it seems that I'm send too much-too fast, if thats possible.
So using the following, (jBuffer) holds json data of about 96 bytes.

client.print(jBuffer); //this works inside a loop that repeats this until all the data fields are sent to the web client

OK works as expected, the client displays the json data as it should. The issue shows up as missing data fields when I send more then 30 sets of data.
So I debug by adding,

client.print(jBuffer);
Serial.print(jBuffer); //show the same json data as was sent to the client

All the data shows up on the serial monitor so I'm certain the data is all being sent. but the client is still missing everything after the 30th field. So I'm thinking maybe it's going too fast for the ethernet to keep up.
So slow it down a bit,

client.print(jBuffer);
delay(25); //slow down the output to ethernet
Serial.print(jBuffer); //show the same json data as was sent to the client

Now that fixed it and all the data is received at the client. Now I don't really want to code hard delays.
My question is does anyone know how to monitor the state of the transmit buffer so I can stop sending to it till theres room for more?
 
What happens if you call client.flush() after sending, instead of that delay? Also, which networking library are you using?
 
I'm working with QNEthernet.h, Good idea I will try that, do you know if client.flush() returns true/false?
 
QNEthernet’s EthernetClient implementation implements the Client interface. The flush() function is defined there. (That’s a clickable link, in case it’s not immediately noticeable.)
The implementation starts here.
(flush() returns void.)

This link might also provide more information:
How to write data to connections
(The “Write immediacy” subsection should explain the TCP delay. Your program doesn’t need a delay, the library just waits a little bit before actually setting the data, in case there’s more. That improves the efficiency for many cases. The flush() call forces an immediate send.)
 
Last edited:
Thanks for the links, I'll look into that info. Do you know where I can find the ethernet transmit buffer size on the t4.1?
 
I was thinking if I know the buffer size I can send that amount of data.
But what make more sense would be to know how much data is waiting to be sent out, and just wait till theres room for more. I will try the writeFully instead of print as stated in the link.
 
writeFully() only solves the problem of ensuring all the data gets sent to the TCP buffers, but it doesn’t necessarily send all data to the network immediately. You’ll still need to use flush(), in addition. (Note that writeFully() may block, if there’s a lot of data.)

I would argue that you don’t really need to know the size of the available write buffer because the plain write(…) calls return the number of bytes actually written. This means you can try to send any count of bytes you’d like, but it will only actually buffer/send what there’s room for, and then return that number. (Having said that, you can use the availableForWrite() call to determine how many bytes can be sent to the stack without blocking.)

This brings us to two styles of programming here: blocking and non-blocking. I almost exclusively use the non-blocking style. (Eg. event- and check-based.) For this, I don’t even often use writeFully(), and instead just track how many bytes I was able to send out in my write(…) calls, and then come back to it in the next task go-around. (I still use writeFully() sometimes, though, if the amount of data I have to send isn’t huge.) There’s use for both. (Note that these two styles don’t just apply to writing data; they also apply to connection, setup, initialization, etc.)
 
Thanks for that info, it will help me solve it I'm sure. I took the code block from the program I was using on the T3.6 with a w5500 module. That worked fine with the print version, how ever that is a different TCP buffer I think.
My idea to try the delay(blocking I know) was based on the T4.1 running 5 times faster then the T3.2.
I will check the write() return function, and implement that as needed.
 
Just to be clear, print() (and write() and writeFully() and all the output functions) will still work, but data won’t get sent until an internal timer expires (about 250ms) or the buffer is full. (This can improve write efficiency.) That’s why you need flush(). Flush() causes data to get sent immediately. Without that call, data will get sent, just a little bit delayed. You don’t need a call to delay(), or even flush(). Data will eventually get sent.

The difference between the QNEthernet library and the one you were using before is that the one you were using before is configured to immediately output the data to the network after all output calls, whilst the QNEthernet library is not configured to do this.

I want to be clear on the point that the libraries behave the same: you don’t need a delay or even a flush call; data will get sent always. There’s just a little timing difference, and to cause data to get sent immediately, call flush().

In other words, to make your old code work, do this:
1. print()
2. Then flush() <- this is the extra bit you need.
 
Thanks shawn for your help, still something for me to look into.
I tried the client.flush() and bingo it worked. However I still found issues when I checked on a slower computer.

So I debug with availableForWrite() before I send more data. With my fast computer the buffer always had room to write as fast as the T4.1 could send it.
The slower comp the buffer would hit 0 before all the data was sent, odd that it would make a difference based on which computer I used.
So I try
Code: 
int rm = client.availableForWrite();
     dbug("room for write --> ");
     dbugln(rm);
               
     if(rm<=1000){        //handle buffer over run when sending data to html
            client.flush();
            dbugln("Tx buffer < 1000");
            delay(250);
     }
my dbug is just Serial.print()

And that seems to work on any computer I try.
The transmit buffer starts out at 5840 and in the Serial window I see the room remaining before I write.
Fast comp it never goes below 1000, slower comp it drops to 0 before all the data is sent.
Still testing.
On the faster comp 9 times out of 10 all the data (45 groups of 189 bytes + html overhead) comes through without hitting the delay(250) and the browser gets all the data within about 10 to 25ms.
the slower comp always hits the delay at about group 22, and without using this delay the availableForWrite goes to zero after the 22nd group, even with the client.flush() after each group.
So for now I'll have to use the delay to make it work, till I can come up with a better way.
 
Last edited:
I wouldn’t use a delay in your code. It also looks much more complicated than it needs to be. You don’t need to manage the library’s buffer.

If you want data to get sent immediately, then flush after writing. If you don’t care if there’s a small delay, then don’t use flush. Data will get sent whether you flush or don’t flush.

Also, the write calls return how much data was actually sent, so you can know what to send the next time around.
 
Last edited:
It would seem that the T4.1 fills the on board ethernet buffer faster then the buffer can empty, Depending on the computer/browser thats requesting the data from the T4.1 server. The client.print/write to the buffer when availableForWrite()=0 is discarded. If I dont check for this part of the data goes missing.
More testing required, I am coding a small program to show this and will post it here soon.
 
Sending data via write() isn’t guaranteed to send all the data. That function returns the number of bytes actually written. That’s one of the motivations I had for adding writeFully(). Either use write() and its return value, or use writeFully(), which keeps trying until all bytes are sent (or there’s an error).
 
You must log in or register to reply here.
Back
Top