Ideas on data transfer

[kriko]

I want to transfer data from PC to teensy 3.0 via usb serial.
In my scenario chunks of data are files resident on PC.

My idea is to wrap data in a package like this:
COMMAND | DATA_LENGTH | DATA

COMMAND in my case is e.g. transfer - 0x01
DATA_LENGTH would be number of bytes to be transferred. This value would be then decreased for every byte read and so when it hits 0, data would be completely transfered.

I'm not sure yet about DATA_LENGTH which type should it be, since chunks of data can be up to 5MB.

Is this idea in general ok, or there are other, better ways to accomplish this?

 

[PaulStoffregen]

This approach can work.

Usually this way is implemented with a timeout, so if the transfer stops, your code can begin looking for "COMMAND" again. The elapsedMillis variable type is probably the simplest way. Usually you would send some message indicating successful reception of the data. The timeout obviously must be longer than the worse-case delay the sender might ever experience while transmitting the data. The downside to this approach is the transmitter much wait longer than the timeout before retrying, if it doesn't hear the success message after sending all the data.

The other common approach is to encode the data so that "COMMAND" can never occur in any valid data pattern. Typically that way also has some EOF (end of file) message which also can't ever occur in the encoded data. The downside is extra complexity to encode the data at the transmitter and decode it (and check for COMMAND and EOF) as you receive all the incoming data.