MTP Responder Contribution

WMXZ · Jan 11, 2021

KurtE said:
Since this is @WMXZ's project, I am trying to figure out how much more that I should do in this pass? My plans is probably to add another command, like 'x' the files that I added, and have versions of command that notifies the host about the files removed and see if that works.

Well, I only adapted the original code to my needs and made it available.
I understand that Paul is interested to integrate it at some time, but he seems to be busy with other (hopefully pleasant) actions.

At the moment, I'm busy to get a 1 TB microSD card working on my data logger.
It is working, but I have from time to time some (probably timeout) issues.

So fee free, to improve it to a level that Paul is willing to take it into TD

Re MTP, The actual version is fine for my (logger) application, where I only need a reset and no dynamic update.
I would be more interested in adding a time stamp to MTP than sending more events.

KurtE · Jan 11, 2021

@WMXZ - Sounds good.
Note: I do think it is good that one can get some of the simple events working for those who might care about it, and in other cases as you mentioned you can use the sledge hammer event of reset, which may be fine for many cases.

So as I mentioned I will probably implement simple file deleted one... and maybe the directory create as I had these working in my other branch.

Time Stamp - I also think it would be a good thing, at least for some File systems. That is I don't think LittleFS has any notion of date/times? Maybe I am wrong, but did not see anything in the stuff I read.

But I do think we should have it as part of the FILE object to support it with SD. However again I am not sure if SDFat/SD have any APIS that export this data? Last time I looked I only found some time code in it for printing the directory type API... Maybe I missed it. Will look again.

In my MTP Host code (USBHost) I did add code to try asking for the Modify Date, which does have a specific format, and the only other device I know I have is an older Kindle Fire that supports MTP, and all of the dates it returned appeared to be some constant date that predates what I thought was the first date... But will investigate more.

In addition, I think it would be fun to have a new test case that maybe shows some other usage cases.

As you mentioned, I think Paul and Robin are very busy right now, but hopefully we can get all of this in a state that it can be part of TD..

KurtE · Jan 11, 2021

@WMXZ/Paul and others - As for MTP and things like Modify date, I put questions and suggestions on the beta 5 thread.

Things like does LittleFS keep any dates anywhere?

FS/FILE - We don't have any APIs defined.

SDFat - I don't think currently there are APIS to retrieve the dates currently only to print them. There is an open Issue on SDFat project on this and mention of he was going to add some for the next release.

Should we we add those proposed APIS to FILE object in FS.h?

If we decide on this I think it would be great to have MTP at least for SDFat like volumes to be able to show their dates on remote machine.

For now will punt, although maybe fun to prototype it in to see...

But first may play with the code again for allowing larger files to be transferred from PC to some slower disks some flash and not timeout...

mjs513 · Jan 11, 2021

@KurtE
Of course it would be nice to have a file creation or file modified date but as you said not sure how LittleFS supports that. I was reading that LFS has custom attributes that you can set and doing a search looks like its done in ESP8266: https://github.com/esp8266/Arduino/pull/7785/files, https://github.com/esp8266/Arduino/pull/7785. Maybe good starting point?

Think getting the timeout resolved is important as well

KurtE · Jan 12, 2021

mjs513 said:
@KurtE
Think getting the timeout resolved is important as well

Me too,

So I am starting to play again of moving over the code I have in other branch.

But also wondering about the differences in the two current versions of the sendObject code:

The one for T3.x is really simple:

Code:

  bool MTPD::SendObject() {
    uint32_t len = ReadMTPHeader();
    while (len) 
    { 
      receive_buffer();
      uint32_t to_copy = data_buffer_->len - data_buffer_->index;
      to_copy = min(to_copy, len);
      if(!storage_->write((char*)(data_buffer_->buf + data_buffer_->index), to_copy)) return false;
      data_buffer_->index += to_copy;
      len -= to_copy;
      if (data_buffer_->index == data_buffer_->len) 
      {
        usb_free(data_buffer_);
        data_buffer_ = NULL;
      }
    }
    storage_->close();
    return true;
  }

It receives a USB buffer and it writes it to the disk... The T4.x version is a little more complicated:

Code:

    bool MTPD::SendObject() 
    { 
      pull_packet(rx_data_buffer);
      read(0,0);
//      printContainer(); 

      uint32_t len = ReadMTPHeader();
      uint32_t index = sizeof(MTPHeader);
      disk_pos=0;
      
      while((int)len>0)
      { uint32_t bytes = MTP_RX_SIZE - index;                     // how many data in usb-packet
        bytes = min(bytes,len);                                   // loimit at end
        uint32_t to_copy=min(bytes, DISK_BUFFER_SIZE-disk_pos);   // how many data to copy to disk buffer
        memcpy(disk_buffer+disk_pos, rx_data_buffer + index,to_copy);
        disk_pos += to_copy;
        bytes -= to_copy;
        len -= to_copy;
        //printf("a %d %d %d %d %d\n", len,disk_pos,bytes,index,to_copy);
        //
        if(disk_pos==DISK_BUFFER_SIZE)
        {
          if(storage_->write((const char *)disk_buffer, DISK_BUFFER_SIZE)<DISK_BUFFER_SIZE) return false;
          disk_pos =0;

          if(bytes) // we have still data in transfer buffer, copy to initial disk_buffer
          {
            memcpy(disk_buffer,rx_data_buffer+index+to_copy,bytes);
            disk_pos += bytes;
            len -= bytes;
          }
          //printf("b %d %d %d %d %d\n", len,disk_pos,bytes,index,to_copy);
        }
        if(len>0)  // we have still data to be transfered
        { pull_packet(rx_data_buffer);
          index=0;
        }
      }
      //printf("len %d\n",disk_pos);
      if(disk_pos)
      {
        if(storage_->write((const char *)disk_buffer, disk_pos)<disk_pos) return false;
      }
      storage_->close();
      return true;
    }

It instead buffers the data internally into its buffer which is currently defined as 8kb and then does a big write. It is trying to write out whole sectors, which depending on the OS, could maybe be faster. That is if the FS does not buffer data internally to do writes...

Again question to self and others: Should FS.h the FS object have some method(s) to ask FS information like: record size, or cluster size, or preferred access size?

But wondering how much of a speed up there is with the buffering and if significant should it be back ported to T3.x...

KurtE · Jan 12, 2021

Quick update to above, and with the current caching code I still get timeouts, depending more on the state of the File writes I think...
Example I did a copy of 341024 bytes long and debug output:

Code:

MTPD::SendObject: len:341024
...............:::::::::::::::::::::::::::::::::::::::+...............+...............:+...............+...............+...............:+...............:+...............+...............+...............:+...............+...............+...............:+...............:+...............+...............+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 316460 0 149453
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 308268 8192 105713
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 300076 16384 102449
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 291884 24576 102932
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 283692 32768 102730
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 275500 40960 102835
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 267308 49152 101551
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 259116 57344 107326
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 250924 65536 107889
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 242732 73728 106393
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 234540 81920 101626
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 226348 90112 101591
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 218156 98304 99576
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 209964 106496 103282
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 201772 114688 103568
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 193580 122880 106592
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 185388 131072 103654
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 177196 139264 101979
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 169004 147456 105982
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 160812 155648 103899
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 152620 163840 99730
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 144428 172032 100119
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 136236 180224 103882
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 128044 188416 102102
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 119852 196608 100042
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 111660 204800 103500
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 103468 212992 101701
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 95276 221184 104119
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 87084 229376 101392
.+.+.+.+...............:WR 8192 8192 85036 237568 97687
:WR 8192 8192 85036 245760 97427
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::..
CL 253952

The :'s for no date went on a long time after hearing the disconnect like chime on windows...

So still not a complete solution.

May hack up a second way, and see what happens if I read the whole file into memory and write it...

WMXZ · Jan 12, 2021

KurtE said:
Again question to self and others: Should FS.h the FS object have some method(s) to ask FS information like: record size, or cluster size, or preferred access size?

But wondering how much of a speed up there is with the buffering and if significant should it be back ported to T3.x...

One application of MTP is to transfer files, while the Teensy is not accessible, so disk access should IMO be optimised.
If this can be done using info from filesystem, so better. Yes, FS.h should provide basic information for user to optimise disk access.

mjs513 · Jan 12, 2021

KurtE said:
Quick update to above, and with the current caching code I still get timeouts, depending more on the state of the File writes I think...
Example I did a copy of 341024 bytes long and debug output:
........
The :'s for no date went on a long time after hearing the disconnect like chime on windows...

So still not a complete solution.

May hack up a second way, and see what happens if I read the whole file into memory and write it...

Just out of curiosity did you try it the way T3.x does it - probably should run a test to see if it works. Stay tuned

EDIT: You probably know this but getting usb event errors on compile for the T3.5"

Code:

C:\Users\Merli\AppData\Local\Temp\arduino_build_145149\libraries\MTP_t4-mtp_events_merge\MTP.cpp.o: In function `MTPD::write(char const*, int)':
D:\Users\Merli\Documents\Arduino\libraries\MTP_t4-mtp_events_merge\src/MTP.cpp:618: undefined reference to `usb_string_serial_number'
C:\Users\Merli\AppData\Local\Temp\arduino_build_145149/..\arduino_cache_106744\core\core_teensy_avr_teensy35_usb_mtp,speed_120,opt_o2std,keys_en-us_2cd4d2f386efe93aad1eadbfdfd194e7.a(usb_dev.c.o): In function `usb_rx_memory':
F:\arduino-1.8.13-beta5\hardware\teensy\avr\cores\teensy3/usb_dev.c:796: undefined reference to `usb_endpoint_config_table'
C:\Users\Merli\AppData\Local\Temp\arduino_build_145149/..\arduino_cache_106744\core\core_teensy_avr_teensy35_usb_mtp,speed_120,opt_o2std,keys_en-us_2cd4d2f386efe93aad1eadbfdfd194e7.a(usb_dev.c.o): In function `usb_setup':
F:\arduino-1.8.13-beta5\hardware\teensy\avr\cores\teensy3/usb_dev.c:195: undefined reference to `usb_descriptor_list'
F:\arduino-1.8.13-beta5\hardware\teensy\avr\cores\teensy3/usb_dev.c:195: undefined reference to `usb_endpoint_config_table'
C:\Users\Merli\AppData\Local\Temp\arduino_build_145149/..\arduino_cache_106744\core\core_teensy_avr_teensy35_usb_mtp,speed_120,opt_o2std,keys_en-us_2cd4d2f386efe93aad1eadbfdfd194e7.a(usb_dev.c.o): In function `usb_init':
F:\arduino-1.8.13-beta5\hardware\teensy\avr\cores\teensy3/usb_dev.c:1170: undefined reference to `usb_init_serialnumber'
collect2.exe: error: ld returned 1 exit status

KurtE · Jan 12, 2021

mjs513 said:
Just out of curiosity did you try it the way T3.x does it - probably should run a test to see if it works. Stay tuned

As I think I posted above, it looks like the T3.x does real simple for USB packet it gets it does a write.

I have not tried that way yet.

I did try some code that, if the file is greater than my bigbuffer it tries to do a extmem_alloc() of the whole file and so far left most of the rest of the code the same, so it does not actually wrap at that point.
I cut the delay between reads to 1ms... And ran it.

Again with the larger file from before it did transfer the whole file but still timed out waiting for me to send back a response.

The debug output:

Code:

MTPD::SendObject: len:341024
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 259628 0 143235
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 209964 8192 97096
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 158764 16384 99778
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 108076 24576 99825
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 57388 32768 98534
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 9260 40960 93950
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+[COLOR="#FF0000"]$[/COLOR]WR 8192 8192 0 49152 97991
WR 8192 8192 0 57344 103734
WR 8192 8192 0 65536 94605
WR 8192 8192 0 73728 99163
WR 8192 8192 0 81920 99886
WR 8192 8192 0 90112 100052
WR 8192 8192 0 98304 101480
WR 8192 8192 0 106496 94145
WR 8192 8192 0 114688 103839
WR 8192 8192 0 122880 100163
WR 8192 8192 0 131072 95135
WR 8192 8192 0 139264 99310
WR 8192 8192 0 147456 102106
WR 8192 8192 0 155648 98636
WR 8192 8192 0 163840 99002
WR 8192 8192 0 172032 98981
WR 8192 8192 0 180224 98646
WR 8192 8192 0 188416 95891
WR 8192 8192 0 196608 99518
WR 8192 8192 0 204800 99510
WR 8192 8192 0 212992 100233
WR 8192 8192 0 221184 99552
WR 8192 8192 0 229376 99777
WR 8192 8192 0 237568 99828
WR 8192 8192 0 245760 98013
WR 8192 8192 0 253952 100298
WR 8192 8192 0 262144 97712
WR 8192 8192 0 270336 97267
WR 8192 8192 0 278528 100637
WR 8192 8192 0 286720 102458
WR 8192 8192 0 294912 101941
WR 8192 8192 0 303104 97371
WR 8192 8192 0 311296 100749
WR 8192 8192 0 319488 100859
WR 8192 8192 0 327680 95343
WR 5152 0 335872 50696
CL 341024

So with the debug output the final $
Shows that it finished reading the file. It then went through the process of writting each of the 8192 blocks.
I wonder if it would make any differences in the timing if instead. of looping calling the write with 8192, I instead called it with the whole count of the data I had available at that time...

But the progress dialog stayed up fully showing all the way to end and then heard the timeout error. I then went back and reset the teensy and was able to open that file as a JPG, so it looks like the data got there.

A possible partial work around might be to have the code send the command completed while still processing the writes. But then not sure what would happen if for example the user had chosen two files to transfer and would it try to start up the second file while I am still doing the writes and would that time out?

I keep looking through MTP document and the google around to see if there are some other work around to this, something like:

a) Maybe some response to send back that says, I am partially done... Don't see one.
b) Or maybe some form of heartbeat event... Again I don't see any, but again don't see anything appropriate. But maybe send the undefined... Maybe it will trigger the host code to say the device must still be there...

c) I don't see any way for the host to send event to device....

KurtE · Jan 12, 2021

Hi again @mjs513 - I have not tried with 3.5, maybe should. I did build for 3.6.

Note: I am building for MTP with using SEREMU... If you are building for MTP + Serial, did you update the usb_descriptor.h (I think that is the file) with the stuff in the modifications for Teensy 3?

Also thinking of taking a minor diversion and play again with my USBHost MTP host test code and maybe try to hack up code to be able to send a file (either a real one or just a bunch of junk) and try it to the Kindle to see if it shows anything interesting during these transfers.

mjs513 · Jan 13, 2021

KurtE said:
Hi again @mjs513 - I have not tried with 3.5, maybe should. I did build for 3.6.

Note: I am building for MTP with using SEREMU... If you are building for MTP + Serial, did you update the usb_descriptor.h (I think that is the file) with the stuff in the modifications for Teensy 3?

Also thinking of taking a minor diversion and play again with my USBHost MTP host test code and maybe try to hack up code to be able to send a file (either a real one or just a bunch of junk) and try it to the Kindle to see if it shows anything interesting during these transfers.

Ok think I am lost on what changes need to be made to the core - think I am behind on some things. Is your core files updated and which branch can I use to be even the MTP changes.

EDIT: Never Mind figure it out.

EDIT2: well got it to compile but can not copy anything to the flash or write to it.

mjs513 · Jan 13, 2021

Morning @KurtE - had my coffee now.

T3.5 and T3.6 both build with no errors. But unless I missed something I can not copy any files to the flash (n01 in this case). However, I can copy from the flash to the desktop, and I can create my logger seems to be working. With that said if I do "reset" it looses the Teensy. Same behavior on both the T3.5/T3.6

KurtE · Jan 13, 2021

Sorry internet down, so by cell...

I noticed this morning that a case statement missing

About line 844 mtp.cpp above sendobjectinfo should be case: ox100c:

Not sure if that is what you are seeing

mjs513 · Jan 13, 2021

KurtE said:
Sorry internet down, so by cell...

I noticed this morning that a case statement missing

About line 844 mtp.cpp above sendobjectinfo should be case: ox100c:

Not sure if that is what you are seeing

Afternoon now for me. So afternoon. Oh man - frustrating when internet is down - been there.

Ok saw where the case statement was missing. Added it in to MTP.cpp and that seems to resolve the issue with copying and deleting files to the T3.6 and probably the T3.5 - haven't tested it yet. But it still has that same problem that we see with the T4.x, i.e., still can not copy that 5.5MB pdf file that is giving me issue. For the T3.6 seems to hang to the hang the T3.6 where we can not do anything else unless i reset it with the pgm button.

Your reset and dump commands now work as well.

KurtE · Jan 13, 2021

Afternoon - Internet back, we are lucky, looking at the power outage map, it looks like a lot of areas are without power.
Wind was really strong overnight. It moved and flipped a trailer we pull behind ATV, plus several smaller trees down... Luckily nothing critical.
Sorry offtopic...

I hear you about the larger files. As I mentioned I think yesterday, when I find that if a transfer takes maybe 5 seconds, windows will time it out and kill the connection. So experimenting.

Currently on diversion of updating my USBHost test program, to maybe try to send a larger file to Kindle Fire and see if it does anything different.

Alternative approach I have thought of, but have not attempted and wonder how well it work and if it would be useful is to have the USBHost code work as a forwarder to MTP code. Basically maybe just forwarding messages that go through the three end points. But maybe later.

Now back to playing

KurtE · Jan 13, 2021

Internet has been up and down... Can not complain too much as it looks like lots are still without power.

I pushed up a WIP branch MEM_send_object_large which is based off of the branch I still have an open PR back to @WMXZ...

It has some of the stuff for a couple of versions of the sendObject to try a few different things like trying go extmem_alloc full size for file... Still times out. As I mentioned I am playing with the other program right now.

But thought I would put it up here. Also while I was playing here, I migrated in a more descriptive printContainer macro.function, which prints out more textual information. Again under #ifdef.

But an example set of output when I did a copy of medium size file:

Code:

Setup done
CMD: 1002(OPEN_SESSION)l: 16 T:0 : 1
CMD: 1001(GET_DEVICE_INFO)l: 12 T:1
CMD: 1014(GET_DEVICE_PROP_DESC)l: 16 T:2 : d402
CMD: 1004(GET_STORAGE_IDS)l: 12 T:3
CMD: 1005(GET_STORAGE_INFO)l: 16 T:4 : 1
CMD: 1005(GET_STORAGE_INFO)l: 16 T:5 : 2
CMD: 1005(GET_STORAGE_INFO)l: 16 T:6 : 3
CMD: 1005(GET_STORAGE_INFO)l: 16 T:7 : 4
CMD: 1005(GET_STORAGE_INFO)l: 16 T:8 : 5
CMD: 9801(GET_OBJECT_PROPS_SUPPORTED)l: 16 T:9 : 3000
CMD: 9801(GET_OBJECT_PROPS_SUPPORTED)l: 16 T:a : 3001
CMD: 1007(GET_OBJECT_HANDLES)l: 24 T:b : 5 0 ffffffff
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:c : e dc02 (FORMAT)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:d : dc03 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:e : e dc03 (PROTECTION)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:f : dc04 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:10 : e dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:11 : e
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:12 : dc01 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:13 : e dc01 (STORAGE_ID)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:14 : dc07 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:15 : e dc07 (OBJECT NAME)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:16 : dc41 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:17 : e dc41 (PERSISTENT_UID)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:18 : dc44 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:19 : e dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1a : d dc02 (FORMAT)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:1b : dc03 3001
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1c : d dc03 (PROTECTION)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:1d : dc04 3001
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:1e : d dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:1f : d
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:20 : dc01 3001
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:21 : d dc01 (STORAGE_ID)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:22 : dc07 3001
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:23 : d dc07 (OBJECT NAME)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:24 : dc41 3001
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:25 : d dc41 (PERSISTENT_UID)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:26 : dc44 3001
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:27 : d dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:28 : c dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:29 : c dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:2a : c dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:2b : c
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:2c : c dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:2d : c dc07 (OBJECT NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:2e : c dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:2f : c dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:30 : b dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:31 : b dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:32 : b dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:33 : b
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:34 : b dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:35 : b dc07 (OBJECT NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:36 : b dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:37 : b dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:38 : a dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:39 : a dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:3a : a dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:3b : a
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:3c : a dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:3d : a dc07 (OBJECT NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:3e : a dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:3f : a dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:40 : 9 dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:41 : 9 dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:42 : 9 dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:43 : 9
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:44 : 9 dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:45 : 9 dc07 (OBJECT NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:46 : 9 dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:47 : 9 dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:48 : 8 dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:49 : 8 dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:4a : 8 dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:4b : 8
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:4c : 8 dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:4d : 8 dc07 (OBJECT NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:4e : 8 dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:4f : 8 dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:50 : 7 dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:51 : 7 dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:52 : 7 dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:53 : 7
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:54 : 7 dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:55 : 7 dc07 (OBJECT NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:56 : 7 dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:57 : 7 dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:58 : 6 dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:59 : 6 dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:5a : 6 dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:5b : 6
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:5c : 6 dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:5d : 6 dc07 (OBJECT NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:5e : 6 dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:5f : 6 dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:60 : 5 dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:61 : 5 dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:62 : 5 dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:63 : 5
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:64 : 5 dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:65 : 5 dc07 (OBJECT NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:66 : 5 dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:67 : 5 dc44 (NAME)
CMD: 100c(SEND_OBJECT_INFO)l: 20 T:68 : 5 ffffffff
DATA:100c(SEND_OBJECT_INFO)l: 176 T:68 : 0 3000 12c8c 3000 0
SendObjectInfo: 5 4294967295 20003500: 0 3000 0 12c8c 3000 0 0 0 2e0 419 18 0 0 0 0 : T4-Card-Front.jpg
CMD: 100d(SEND_OBJECT)l: 12 T:69
MTPD::SendObject: len:76940
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+$WR 8192 8192 0 0 145804
WR 8192 8192 0 8192 101829
WR 8192 8192 0 16384 105095
WR 8192 8192 0 24576 99170
WR 8192 8192 0 32768 101161
WR 8192 8192 0 40960 99761
WR 8192 8192 0 49152 104204
WR 8192 8192 0 57344 99237
WR 8192 8192 0 65536 103125
WR 3212 0 73728 5392
CL 76940
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:6a : f dc02 (FORMAT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:6b : f dc01 (STORAGE_ID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:6c : f dc07 (OBJECT NAME)
CMD: 9802(GET_OBJECT_PROP_DESC)l: 20 T:6d : dc0b 3000
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:6e : f dc0b (PARENT)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:6f : f dc41 (PERSISTENT_UID)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:70 : f dc44 (NAME)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:71 : f dc03 (PROTECTION)
CMD: 9803(GET_OBJECT_PROP_VALUE)l: 20 T:72 : f dc04 (SIZE)
CMD: 1008(GET_OBJECT_INFO)l: 16 T:73 : f
CMD: 1005(GET_STORAGE_INFO)l: 16 T:74 : 5

I will probably extend the messages that print out which property... But for me this makes it little easier to see what is happening.

Note: the .+ like stuff is some debug stuff for the send Object Where you can see, by the time the first write completed it had received the majority of the file over USB. The last number in the WR lines is the number of micros that write operation took. On average looks like about a tenth of second for 8K...

KurtE · Jan 15, 2021

Another quick update:

I have been playing around with my USBHost MTP code and communicating with Kindle Fire. Note: The connection with Kindle Fire (Android) is a bit touchy. Sometimes works other times get into endless start connection drop...

But I hacked up some code in my simple sketch (part of USBHost_t36 fork) MTP_Device branch.

I added some code to allow me to send a file currently from SDCard on my T4.1 to the Kindle to see if I get any unexpected messages or the like. So far not.

In case any one is interested: here is some of the terminal output... I removed most of the debug messages:

Code:

========== Internal storage ===========
Music(1) FMT:3001 Size:0 Modify:19691231T160000
Podcasts(2) FMT:3001 Size:0 Modify:19691231T160000
Ringtones(3) FMT:3001 Size:0 Modify:19691231T160000
Alarms(4) FMT:3001 Size:0 Modify:19691231T160000
Notifications(5) FMT:3001 Size:0 Modify:19691231T160000
Pictures(6) FMT:3001 Size:0 Modify:19691231T160000
Movies(7) FMT:3001 Size:0 Modify:19691231T160000
Download(8) FMT:3001 Size:0 Modify:19691231T160000
DCIM(9) FMT:3001 Size:0 Modify:19691231T160000
Android(a) FMT:3001 Size:4096 Modify:19691231T160000
kindle(3a) FMT:3001 Size:4096 Modify:19691231T160000
Books(3b) FMT:3001 Size:4096 Modify:19691231T160000
Documents(3e) FMT:3001 Size:0 Modify:19691231T160000
.imagecache(40) FMT:3001 Size:0 Modify:19691231T160000
Audible(44) FMT:3001 Size:0 Modify:19691231T160000
Contacts(b0) FMT:3001 Size:0 Modify:19691231T160000
.bugsense(272) FMT:3000 Size:33 Modify:19691231T160000
mapquest(276) FMT:3001 Size:0 Modify:19691231T160000
sirius(421) FMT:3001 Size:4096 Modify:19691231T160000
TuneIn Radio(5e6) FMT:3001 Size:4096 Modify:19691231T160000
amazonmp3(925) FMT:3001 Size:0 Modify:19691231T160000
AccuWeather(107c) FMT:3001 Size:0 Modify:19691231T160000
.backups(115c) FMT:3001 Size:0 Modify:19691231T160000
.chartboost(1160) FMT:3001 Size:0 Modify:19691231T160000
abbba(1508) FMT:3000 Size:0 Modify:19691231T160000
/storage/emulated/0/(1574) FMT:3000 Size:1000 Modify:19691231T160000
T4.1-Cardlike(158f) FMT:3801 Size:393481 Modify:19691231T160000
SendFile called
Command SendFile: 10001 IMG_1258.jpg
C-> CMD: 100c(SEND_OBJECT_INFO)l:20 T:121 P:10001 3e
d-> DATA:100c(SEND_OBJECT_INFO)l:94 T:121 P:10001 3000 41cf6 0 0
5E 00 00 00 02 00 0C 10 21 01 00 00 01 00 01 00 00 30 00 00 F6 1C 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 3E 00 00 00 00 00 00 00 00 00 00 00 00 00 0D 49 00 4D 00 47 00 5F 00 31 00 32 00 35 00 38 00 2E 00 6A 00 70 00 67 00 00 00 00 00 00 
sendObjectMsg 1592
C-> CMD: 100d(SEND_OBJECT)l:12 T:122
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT

MTP_OPERATION_SEND_OBJECT  completed new Object: 1592  1592 - node allocated(70000ec4)
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:123 P:1592 dc01 (STORAGE_ID)
processGetObjectPropValue(STORAGE_ID): 70000ec4 0 dc01: 1 0 1 0
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:124 P:1592 dc02 (FORMAT)
processGetObjectPropValue(OBJECT_FORMAT): 70000ec4 1 dc02: 1 38 1 0
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:125 P:1592 dc03 (PROTECTION)
processGetObjectPropValue(PROTECTION_STATUS): 70000ec4 2 dc03: 0 0 1 0
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:126 P:1592 dc04 (SIZE)
processGetObjectPropValue(OBJECT_SIZE): 70000ec4 3 dc04: f6 1c 4 0
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:127 P:1592 dc07 (OBJECT NAME)
processGetObjectPropValue(OBJECT_FILE_NAME): 70000ec4 4 dc07: d 49 0 4d
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:128 P:1592 dc09 (MODIFIED)
processGetObjectPropValue(DATE_MODIFIED): 70000ec4 5 dc09: 10 31 0 39
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:129 P:1592 dc0b (PARENT)
processGetObjectPropValue(PARENT_OBJECT): 70000ec4 6 dc0b: 3e 0 0 0
    Parent set: 70000504
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:12a P:1592 dc41 (PERSISTENT_UID)
processGetObjectPropValue(PERSISTENT_UID): 70000ec4 7 dc41: 92 15 0 0
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:12b P:1592 dc44 (NAME)
processGetObjectPropValue(NAME): 70000ec4 8 dc44: 9 49 0 4d
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:12c P:1592 dc4e
processGetObjectPropValue(): 70000ec4 9 dc4e: 10 31 0 39
mtpd_event_callback 4002 1592 70000ec4 : 0 0 70000504
 CALLBACK: Object added: 70000ec4 ID:00001592 P:70000504 C:0000: S:00010001 F:3801 MD: 19691231T160000 IMG_1258


 ================= Start ENUM(62:Documents) =================
C-> CMD: 1007(GET_OBJECT_HANDLES)l:24 T:12d P:10001 0 3e
  3f - node allocated(70000f48)
  6b4 - node allocated(70000fa8)
  752 - node allocated(70001008)
  14fd - node allocated(70001068)
  1502 - node allocated(700010c8)
  1505 - node allocated(70001128)
  1590 - node allocated(70001188)
  1592 - node reused
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:12e P:3f dc01 (STORAGE_ID)
processGetObjectPropValue(STORAGE_ID): 70000f48 0 dc01: 1 0 1 0
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:12f P:3f dc02 (FORMAT)
processGetObjectPropValue(OBJECT_FORMAT): 70000f48 1 dc02: 1 30 1 0
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:130 P:3f dc03 (PROTECTION)
processGetObjectPropValue(PROTECTION_STATUS): 70000f48 2 dc03: 0 0 1 0
...
C-> CMD: 9803(GET_OBJECT_PROP_VALUE)l:20 T:17d P:1592 dc4e
processGetObjectPropValue(): 70000ec4 9 dc4e: 10 31 0 39

========== Enum completed (62:Documents ===========
Sidecars(3f) FMT:3001 Size:4096 Modify:19691231T160000
PSNL!DigitalSupportPSNLs!bWFrZVNvZnR3YXJlVXBncmFkZUxldHRlci5odG1sLz9sb2NhbGU9QVRWUERLSUtYMERFUg_PSNL(6b4) FMT:3000 Size:88192 Modify:19691231T160000
PSNL!DigitalSupportPSNLs!bWFrZVdhUG9MZXR0ZXIuaHRtbD9iaW5hcnk9bW9iaTg_PSNL(752) FMT:3000 Size:81760 Modify:19691231T160000
Naxos.xls(14fd) FMT:ba85 Size:11776 Modify:19691231T160000
qwerty(1502) FMT:3000 Size:0 Modify:19691231T160000
new_file(1505) FMT:3000 Size:0 Modify:19691231T160000
T4.1-Cardlike(1590) FMT:3801 Size:393481 Modify:19691231T160000
IMG_1258(1592) FMT:3801 Size:269558 Modify:19691231T160000
20007470 ID:00010001 P:0000 C:70000084: S:00010001 F:3001 MD:  Internal storage
70000084   ID:00000001 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Music
700000e4   ID:00000002 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Podcasts
70000144   ID:00000003 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Ringtones
700001a4   ID:00000004 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Alarms
70000204   ID:00000005 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Notifications
70000264   ID:00000006 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Pictures
700002c4   ID:00000007 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Movies
70000324   ID:00000008 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Download
70000384   ID:00000009 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 DCIM
700003e4   ID:0000000a P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Android
70000444   ID:0000003a P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 kindle
700004a4   ID:0000003b P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Books
70000504   ID:0000003e P:20007470 C:70000f48: S:00010001 F:3001 MD: 19691231T160000 Documents
70000f48     ID:0000003f P:70000504 C:0000: S:00010001 F:3001 MD: 19691231T160000 Sidecars
70000fa8     ID:000006b4 P:70000504 C:0000: S:00010001 F:3000 MD: 19691231T160000 PSNL!DigitalSupportPSNLs!bWFrZVNvZnR3YXJlVXBncmFkZUxldHRlci5odG1sLz9sb2NhbGU9QVRWUERLSUtYMERFUg_PSNL
70001008     ID:00000752 P:70000504 C:0000: S:00010001 F:3000 MD: 19691231T160000 PSNL!DigitalSupportPSNLs!bWFrZVdhUG9MZXR0ZXIuaHRtbD9iaW5hcnk9bW9iaTg_PSNL
70001068     ID:000014fd P:70000504 C:0000: S:00010001 F:ba85 MD: 19691231T160000 Naxos.xls
700010c8     ID:00001502 P:70000504 C:0000: S:00010001 F:3000 MD: 19691231T160000 qwerty
70001128     ID:00001505 P:70000504 C:0000: S:00010001 F:3000 MD: 19691231T160000 new_file
70001188     ID:00001590 P:70000504 C:0000: S:00010001 F:3801 MD: 19691231T160000 T4.1-Cardlike
70000ec4     ID:00001592 P:70000504 C:0000: S:00010001 F:3801 MD: 19691231T160000 IMG_1258
70000564   ID:00000040 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 .imagecache
700005c4   ID:00000044 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Audible
70000624   ID:000000b0 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 Contacts
70000684   ID:00000272 P:20007470 C:0000: S:00010001 F:3000 MD: 19691231T160000 .bugsense
700006e4   ID:00000276 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 mapquest
70000744   ID:00000421 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 sirius
700007a4   ID:000005e6 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 TuneIn Radio
70000804   ID:00000925 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 amazonmp3
70000864   ID:0000107c P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 AccuWeather
700008c4   ID:0000115c P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 .backups
70000924   ID:00001160 P:20007470 C:0000: S:00010001 F:3001 MD: 19691231T160000 .chartboost
70000984   ID:00001508 P:20007470 C:0000: S:00010001 F:3000 MD: 19691231T160000 abbba
700009e4   ID:00001574 P:20007470 C:0000: S:00010001 F:3000 MD: 19691231T160000 /storage/emulated/0/
70000a44   ID:0000158f P:20007470 C:0000: S:00010001 F:3801 MD: 19691231T160000 T4.1-Cardlike
port change: 1C00100A
    disconnect

It is interesting that all of the files have the date: 19691231T160000

I also see those dates also on the Kindle although there are some files created there that have an actual date.

Next up, will try another pass at the SEND_OBJECT code in the MTP proper. I am thinking of trying simple version, that once it reads all of the stuff into the large buffer, it returns and then maybe background finish doing writes of the file until complete and then free the buffer...

KurtE · Jan 16, 2021

I know ... talking to myself

Just pushed up a WIP version of the MTP send object code, that in addition of trying to allocate a full size buffer for the file. Once the whole file has been received over USB, I will exit the method and have the code return a success status to the host. Then in calls to MTPD::loop it will see we have outstanding writes to do and issue one buffer per call...

When the transfer completes, it will send an event to host saying the size changed as during this time the item will be marked as having a 0 length. Doing F5 does show up the new proper size.

Again WIP - I will probably now relax this slightly and maybe only resort to this when the total time to process the file is > some timeout like 3 or so seconds.

Not sure if any good way to show the user that it is still active writing the file.

With this I did transfer a 3MB image to the really slow QSPI drive...

PaulStoffregen · Jan 16, 2021

Merging and testing MTP and LittleFS pull requests is at the top of my priority list for after the T4 bootloader chip “soft” release. Sorry I can’t do it now... really want to. But so many people have been waiting for those chips, so I must get that done.

mjs513 · Jan 16, 2021

PaulStoffregen said:
Merging and testing MTP and LittleFS pull requests is at the top of my priority list for after the T4 bootloader chip “soft” release. Sorry I can’t do it now... really want to. But so many people have been waiting for those chips, so I must get that done.

Fully understandable - don't get distracted at this point sounds like you are almost there with the bootloader. As you said think you are going to make a lot of folks happy when the bootloader becomes available.

mjs513 · Jan 16, 2021

KurtE said:
I know ... talking to myself

Just pushed up a WIP version of the MTP send object code, that in addition of trying to allocate a full size buffer for the file. Once the whole file has been received over USB, I will exit the method and have the code return a success status to the host. Then in calls to MTPD::loop it will see we have outstanding writes to do and issue one buffer per call...

When the transfer completes, it will send an event to host saying the size changed as during this time the item will be marked as having a 0 length. Doing F5 does show up the new proper size.

Again WIP - I will probably now relax this slightly and maybe only resort to this when the total time to process the file is > some timeout like 3 or so seconds.

Not sure if any good way to show the user that it is still active writing the file.

With this I did transfer a 3MB image to the really slow QSPI drive...

Just gave the updated branch a try with the NAND 1G chip on SPI, figured that would be the slowest transfers. Couldn't seem to transfer a PDF over 2MB to the chip. However, restoring back to mtp_events_merge branch worked successfully up to a 19MB PDF. Only 1 file seems to be giving me an issue and maybe its just an issue with that file.

On the 25Q64JV I was able to transfer a 214KB JPG and a 4.76KB TXT file but a 1.1MB PDF failed and had to do a restart of the T4.0. EDIT: Same behavior on QSPI.

On the 25Q128JV I was able to transfer PDFs up to 4.8MB and a zip file of 5.1MB

I can test these 2 with your new branch if you would like as well.

defragster · Jan 16, 2021

PaulStoffregen said:
Merging and testing MTP and LittleFS pull requests is at the top of my priority list for after the T4 bootloader chip “soft” release. Sorry I can’t do it now... really want to. But so many people have been waiting for those chips, so I must get that done.

That's great. When the version to release is assembled I'll look to get back and test it out when the code to be included is known and in one place.

Having the 1062 bootloader chip ready for the test pass will be awesome to have out there so builders can get busy.

KurtE · Jan 17, 2021

@Paul - sounds great

@mjs513 - Did you have enough memory PSRAM available to be able to allocated that large of a file? Right now I try to allocate the whole thing with extmem_alloc. So if that is not available it falls back to using internal buffer and as such to read into the 3 times write size... Two differences I know between two branches... I believe we reduced the size of writes in the other branch.

I right now try to read as much faster now. In other branch I tried to pace it more... Maybe I will try experiment of doing some more pacing.

Wish it was clear on knowing when Windows will decide to error out the whole connection...

mjs513 · Jan 17, 2021

KurtE said:
@mjs513 - Did you have enough memory PSRAM available to be able to allocated that large of a file? Right now I try to allocate the whole thing with extmem_alloc. So if that is not available it falls back to using internal buffer and as such to read into the 3 times write size... Two differences I know between two branches... I believe we reduced the size of writes in the other branch.

@KurtE = Good Sunday Morning.

Well been testing on a T4.0 and a T4.1 with No PSRAM chips installed. Didn't sink in last night you were using PSRAM last night. Was doing some more testing this morning with the T4.0 - no PSRAM - will retest with the PSRAM but wont be able to test QSPI.
@KurtE
Reinstalled your mem_send_object branch (think thats the latest) and the one you are referring to and retested the FLASH chips on SPI with a T4.0:

On the 25Q64JV:
1. Failed to transfer a 1.1MB PDF:

Code:

CMD: 100c(SEND_OBJECT_INFO)l: 20 T:20 : 1 ffffffff
DATA:100c(SEND_OBJECT_INFO)l: 172 T:20 : 0 3000 11a470 3000 0
SendObjectInfo: 1 4294967295 20002920: 0 3000 0 11a470 3000 0 0 0 0 0 0 0 0 0 0 : aoyue-8032a.pdf
RESP:2001(RSP:OK)l: 24 T:20 : 1 ffffffff 3
CMD: 100d(SEND_OBJECT)l: 12 T:21
MTPD::SendObject: len:1156208
...............::::::::::::::::::::::::::::::::::::::+...........+...........+...............:+..........+...........+...........+............+...........+...........+...............:+...........+...........+...........+..........+............+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1137788 0 21666
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1128060 8192 20353
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1118332 16384 20500
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1109628 24576 20434
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1099900 32768 20456
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1090684 40960 20495
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1082492 49152 20378
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1074300 57344 20663
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1066108 65536 20430
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1057916 73728 20245
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1049724 81920 20252
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1041532 90112 20633
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1033340 98304 20344
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1025148 106496 20505
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1016956 114688 20166
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1008764 122880 20555
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 1000572 131072 20688
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 992380 139264 20414
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 984188 147456 58080
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 975996 155648 20510
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 967804 163840 20060
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 959612 172032 20462
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 951420 180224 20280
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 943228 188416 20611
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 935036 196608 20175
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 926844 204800 20422
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 918652 212992 20289
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 910460 221184 20472
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 902268 229376 20211
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 894076 237568 20277
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 885884 245760 20252
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 877692 253952 20644
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 869500 262144 58576
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 861308 270336 20520
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 853116 278528 20461
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 844924 286720 20583
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 836732 294912 20464
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 828540 303104 20460
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 820348 311296 20239
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 812156 319488 20495
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 803964 327680 20419
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 795772 335872 20371
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 787580 344064 20435
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 779388 352256 20563
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 771196 360448 20323
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 763004 368640 20105
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 754812 376832 20405
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 746620 385024 20657
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 738428 393216 20948
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 730236 401408 20339
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 722044 409600 20370
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 713852 417792 20485
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 705660 425984 101633
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 697468 434176 102008
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 689276 442368 98328
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 681084 450560 103983
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 672892 458752 98611
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 664700 466944 98897
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 656508 475136 99708
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 648316 483328 109136
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 640124 491520 99658
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 631932 499712 96292
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 623740 507904 106668
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 615548 516096 98932
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 607356 524288 105910
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 599164 532480 96793
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 590972 540672 95708
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 582780 548864 98634
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 574588 557056 99421
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 566396 565248 94050
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 558204 573440 96092
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 550012 581632 99220
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 541820 589824 99822
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 533628 598016 94748
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 525436 606208 95118
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 517244 614400 98192
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 509052 622592 94890
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 500860 630784 95495
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 492668 638976 99574
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 484476 647168 103460
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 476284 655360 94784
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 468092 663552 101519
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 459900 671744 97962
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 451708 679936 99993
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 443516 688128 99767
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 435324 696320 96837
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 427132 704512 93062
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 418940 712704 97535
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 410748 720896 93621
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 402556 729088 96557
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 394364 737280 95210
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+WR 8192 8192 386172 745472 100852
.+.+.+.+...............::WR 8192 8192 384124 753664 101326
:WR 8192 8192 384124 761856 92346

:::::::::::::::::::::::::::::::::::::::


*** tranfer did not complete fully ***
CL 770048
RESP:2005(RSP:OPERATION_NOT_SUPPORTED)l: 12 T:21
WRLE 2036 384124 770048 0
CL 772084

and it lost the flash drive. Doing a reset did not recover the drive. Restarting showed the file was there but was only 725kb
2. Again was able to transfer the jpg and txt documents no problem.
3. Tested on QSPI and same behaviour was observed.

On the 25Q128JV:
1. Was able to transfer the 1.1MB PDF successfully this time as well as the jpg and txt.
2. Was able to transer a 2.5MB csv file
3. was able to transfer a 3.4 MB pdf file
4. was able to transfer a 4.5 MB pdf file
5. Failed on 19mb pdf file

EDIT: may be using the wrong respository.

KurtE · Jan 17, 2021

Morning - Right now playing in the branch: MEM_send_object_large

MTP Responder Contribution

Well-known member

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Well-known member

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Well-known member

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+

Senior Member+