Forum Rule: Always post complete source code & details to reproduce any issue!
Page 23 of 23 FirstFirst ... 13 21 22 23
Results 551 to 569 of 569

Thread: Many TLAs: MTP MSC FS SD SDFat LittleFS UsbMSCFat to work with each other 8)

  1. #551
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    26,398
    I used the original repository's default branch. Haven't looked at any other branches (yet). I tested against all the example programs on the main branch of the original UsbMscFat.

    The 1 and only branch I've created was meant to preserve a copy before I deleted lots of stuff.

    Looks like I missed a bunch of stuff on branches. Don't panic. It can all get merged, though I do want to carefully consider how to do so. Even if it takes a while longer to restructure somewhat, I want to limit how far our SdFat fork diverges from Bill's original code.

    Another question is how much of this stuff really must get merged before we start 1.57-beta1? Some major improvements have happened since 1.56, like Wire library slave mode, Audio ADC usable performance, and of course the SdFat 2.1.0 -> 2.1.2 to fix large file append. More can always be merged for 1.57-beta2. Getting this to a usable point is currently the thing holding up beginning 1.57 beta testing.

  2. #552
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    10,547
    Quote Originally Posted by PaulStoffregen View Post
    Another question is how much of this stuff really must get merged before we start 1.57-beta1? Some major improvements have happened since 1.56, like Wire library slave mode, Audio ADC usable performance, and of course the SdFat 2.1.0 -> 2.1.2 to fix large file append. More can always be merged for 1.57-beta2. Getting this to a usable point is currently the thing holding up beginning 1.57 beta testing.
    Obviously it is your decision on when you wish to release a new beta. If There is lots of stuff that others are needing, one option would be to simply release the beta without any of the MSC stuff.

    Quote Originally Posted by PaulStoffregen View Post

    Looks like I missed a bunch of stuff on branches. Don't panic. It can all get merged, though I do want to carefully consider how to do so. Even if it takes a while longer to restructure somewhat, I want to limit how far our SdFat fork diverges from Bill's original code.
    Understood. I am not panicking, but for now I will hold off and work on some different stuff.

  3. #553
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    8,236
    Quote Originally Posted by PaulStoffregen View Post
    Another question is how much of this stuff really must get merged before we start 1.57-beta1? Some major improvements have happened since 1.56, like Wire library slave mode, Audio ADC usable performance, and of course the SdFat 2.1.0 -> 2.1.2 to fix large file append. More can always be merged for 1.57-beta2. Getting this to a usable point is currently the thing holding up beginning 1.57 beta testing.
    As to the question of releasing 1.57 Beta-1. As Kurt said the final decision is yours but right now you probably have a base for MSC that you could go ahead and release but am concerned that some of the MTP_Teensy (our version) and MTP_T4 will probably be broken or limited in use. Haven't really checked yet but just a gut feeling.

    EDIT: On a second thought maybe should leave the MSC stuff out for Beta-1. Without all the other pieces may cause more problems than its worth.

    Quote Originally Posted by PaulStoffregen View Post

    Looks like I missed a bunch of stuff on branches. Don't panic. It can all get merged, though I do want to carefully consider how to do so. Even if it takes a while longer to restructure somewhat, I want to limit how far our SdFat fork diverges from Bill's original code.
    Again as Kurt said not panicking but kind of stuck for bit with using MTP which is really where MSC was coming in handy and we do use.
    Last edited by mjs513; 05-04-2022 at 04:56 PM. Reason: EDITED COMMENTS

  4. #554
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    26,398
    It's kinda late to turn back now, kind of like swimming half way across a river.

    I'll plan on waiting a few more days to package up 1.57-beta1.

  5. #555
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    26,398
    Ok, after much thought and research and experimenting yesterday, I decided to modify SdFat FsVolume begin() so we can explicitly tell SdFat the location of the partition to use.

    https://github.com/PaulStoffregen/Sd...c8a14c5a683e87

    The idea is instead of telling FsVolume begin() which partition, we tell it the starting sector and number of sectors to use. So far I've only done FsVolume, but I'm planning to do the same for format.

    This gives USBHost_t36 the responsibility to determine where the partitions are located. For now, I committed a very simple MBR-only function.

    https://github.com/PaulStoffregen/US...2d573eee9b3c31

    Of course, the plan is to bring all the GPT stuff into these functions in USBHost_t36. At this moment all of my USB drives have only MBR with only 1 partition. Later today I'll set up a couple drives with complex partitioning.

    While this is a structural change from how everything has been developed so far inside PFsLib, the idea is to leave SdFat basically as Bill intended. It should (hopefully) result in a "cleaner" patch where we add alternate begin() & init() functions, and except for the 2TB capacity limit, we shouldn't need to keep changing stuff inside SdFat.

  6. #556
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    8,236
    A lot of work Paul but sounds like you got the final structure you want to use which is good. Let us know when you want us to test.

    Of course, the plan is to bring all the GPT stuff into these functions in USBHost_t36. At this moment all of my USB drives have only MBR with only 1 partition. Later today I'll set up a couple drives with complex partitioning.
    You mean like this:
    Click image for larger version. 

Name:	IMG-0637.jpg 
Views:	2 
Size:	73.7 KB 
ID:	28289

  7. #557
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    10,547
    Quick update...

    Things are coming along Probably a few more pieces to be put into place, which I am guessing as it has been progressing that Paul has been filtering in the changes.


    Right not playing around with Test sketch where I plug in a CircuitPython board into the USBHost board that has MTP setup to forward the MSC stuff, plus stuff in place to also talk to the USBSerial object.

    I am mostly now testing using the not official MTP+Serial as a I am also glutton for punishment and using IDE2 some of the time and currently still does not support SEREMU.

    For anyone else who might try this on Linux... One thing I ran into was MTP would work fine with MTP_DISK but not MTP_SERIAL... Ubuntu would not open up MTP window.
    Yet the MTP library code said it was a valid MTP drive...

    Long Story Short:
    Found out, that for whatever reason the udev rules (00-teensy.rules) was setup to stop it from working...
    In particular this line: ATTRS{idVendor}=="16c0", ATTRS{idProduct}=="04*", ENV{ID_MM_DEVICE_IGNORE}="1", ENV{ID_MM_PORT_IGNORE}="1"

    After I modified that line, like magic it worked.

    Now back to figuring out why the USBHOst code and MPy code is not getting all of the Python Serial output... Not that the Serial code is losing it... We are not getting any USB traffic for these messages.
    Yet others come through fine... Not sure which side it is...

  8. #558
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    8,236
    Ditto here on playing with the latest changes from this morning.

    Mainly been playing with the MTP Integrity sketch (kitchen sink version) and it seems to be working with some caveats. As Kurt said been using the unoffocial MTP+Serial (hopefully eventually official).

    The only modification made to the sketch, besides the obvious change to class names was to use Kurt's updated checkMSCChanges() function which provides a hack for now of only setting up the first volume for each USB drive. Reason: the bool operator on the USBFileSystem object has not been added yet nor way to delete/release them so this is just a preliminary test of MTP + USB changes. Yes I can add and delete files - formatting test is tbd as we know so didnt try....

    Code:
    @@@@@@@@@@@@@@@ NEW Drives @@@@@@@@@@@
    
    ##USB Drive: 0 connected
    
    Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    *** GPT Disk WIP ***
    GPT guard:	1,0,0x0,0x2,0x0,0xEE,0xFE,0xBF,0xDB,1,4294967295
    pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    	 < unused area starting at: 0 length 242075647 >
    
    GPT partition header revision: 10000
    LBAs current:1 backup:242075647 first:34 last:242075614
    Disk GUID:494E6C4B-0928-4119-6EED-8BD052C8A7BAStart LBA Array: 2 Count: 128 size:128
    Part	 Type Guid, Unique Guid, First, last, attr, name
    0	EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, 466E5E58-6453-84BB-6CDD-3B813E5167A3, 2048, 60520447, 0, 
    >>> Microsoft Basic Data Partition
    1	EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, 08F18C4C-41DD-48E2-2784-EFD893305C7C, 60520448, 242073599, 0, 
    >>> Microsoft Basic Data Partition
        EXFAT:
    3	00000000-0000-0000-0000-000000000000, 00000000-0000-0000-0000-000000000000, 0, 0, 0, 
    Found new Volume:0
    
    @@@@@@@@@@@@@@@ NEW Drives @@@@@@@@@@@
    
    ##USB Drive: 1 connected
    
    Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,47298560
    Extend:	2,0,0xFE,0xFF,0xFF,0xF,0xFE,0xFF,0xFF,47300608,73844736
    exFAT:	2:1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,47300672(64),73844672 (0)
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    Found new Volume:1
    
    Dump Storage list(7)
    .....
    store:5 storage:60001 name:USB0-GPTxFat fs:20011b0c pn:
    store:6 storage:70001 name:USB1-FAT32-P fs:20011fa0 pn:
    I also attached a SSD drive and that worked as well.

  9. #559
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    26,398
    I'm leaning towards packaging up 1.57-beta1 this afternoon. We're far from finished for a final 1.57 release, but I hope we're far enough along to make this and so many other non-filesystem improvements available to everyone willing to run a beta installer (but probably not cobble together many github downloads not knowing which versions & branches are meant to be dependencies).

    Any change to the USB Type list involve a bunch of thorny issues, especially with Linux and udev rules. I do intend to add a MTP+Serial before final 1.57 release. But I'm not eager to do all that testing right now, holding up a 1.57-beta1 installer.

    The main point a 1.57-beta1 is to give people who want to join in testing & using all this new stuff an accessible way to get it installed with all dependencies correct.

  10. #560
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    8,236
    Quote Originally Posted by PaulStoffregen View Post
    I'm leaning towards packaging up 1.57-beta1 this afternoon. We're far from finished for a final 1.57 release, but I hope we're far enough along to make this and so many other non-filesystem improvements available to everyone willing to run a beta installer (but probably not cobble together many github downloads not knowing which versions & branches are meant to be dependencies).

    Any change to the USB Type list involve a bunch of thorny issues, especially with Linux and udev rules. I do intend to add a MTP+Serial before final 1.57 release. But I'm not eager to do all that testing right now, holding up a 1.57-beta1 installer.

    The main point a 1.57-beta1 is to give people who want to join in testing & using all this new stuff an accessible way to get it installed with all dependencies correct.
    MTP+Serial: Understood and understandable. Will be nice when 1.57 is released and to be honest didn't think it would be in for beta1 testing.

    Guess for initial beta1 release its a great start. A lot of work put in to getting it all incoporated. Think for the beta2 release probably should try and get in formatting stuff and get the partition enumeration (did I say that right). Maybe good idea to add a note on limitation/capability for the beta release so people know what to expect. Without those 2 things may break some things people have already implemented in MTP if they are using partitioned drives or with USBMscFat.
    Last edited by mjs513; 05-06-2022 at 06:07 PM. Reason: EDIT

  11. #561
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,451
    One issue with MTP+Serial is that it uses for the moment the PID of Everything
    Even if MTP+Serual is not yet ready to be included for full, I continue to suggest that Paul defines a dedicated PID, so we can run the test with out own implementations.
    I ONLY use MTP+Serial, but I'm on windows using Putty to communicate with Teensy and to log the Serial.print to disk.

  12. #562
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    10,547
    Quote Originally Posted by PaulStoffregen View Post
    I'm leaning towards packaging up 1.57-beta1 this afternoon. We're far from finished for a final 1.57 release, but I hope we're far enough along to make this and so many other non-filesystem improvements available to everyone willing to run a beta installer (but probably not cobble together many github downloads not knowing which versions & branches are meant to be dependencies).

    Any change to the USB Type list involve a bunch of thorny issues, especially with Linux and udev rules. I do intend to add a MTP+Serial before final 1.57 release. But I'm not eager to do all that testing right now, holding up a 1.57-beta1 installer.

    The main point a 1.57-beta1 is to give people who want to join in testing & using all this new stuff an accessible way to get it installed with all dependencies correct.
    Likewise understood. Although my guess is that many of us have done more testing using +Serial than with +Seremu...

    The issue I mentioned, sort of stumped me for a bit, but then by chance looked at the udev rules.

    I don't know what ranges of PID's you have available, but with:
    Code:
    #elif defined(USB_MTPDISK)
      #define VENDOR_ID		0x16C0
      #define PRODUCT_ID		0x04D1
    ...
    And if it was available:
    If instead of current 0x476 which was the everything PID
    We could use 0x4D2 I am pretty sure your udev rules would not have to change.

    like:
    Code:
    #elif defined(USB_MTPDISK_SERIAL)
      #define VENDOR_ID             0x16C0
      #define PRODUCT_ID            0x04D2  // New MTP + Serial
      #define RAWHID_USAGE_PAGE 0xFFAB  // recommended: 0xFF00 to 0xFFFF
      #define RAWHID_USAGE    0x0200  // recommended: 0x0100 to 0xFFFF
      #define DEVICE_CLASS    0xEF
      #define DEVICE_SUBCLASS 0x02
      #define DEVICE_PROTOCOL 0x01
    
      #define MANUFACTURER_NAME     {'T','e','e','n','s','y','d','u','i','n','o'}
      #define MANUFACTURER_NAME_LEN 11
      #define PRODUCT_NAME          {'T','e','e','n','s','y',' ','M','T','P',' ','D','i','s','k','/','S','e','r','i','a','l'}
      #define PRODUCT_NAME_LEN      22
      #define EP0_SIZE    64
    
      #define NUM_ENDPOINTS         5
      #define NUM_INTERFACE   3
      #define CDC_IAD_DESCRIPTOR  1
      #define CDC_STATUS_INTERFACE  0
      #define CDC_DATA_INTERFACE  1 // Serial
      #define CDC_ACM_ENDPOINT  2
      #define CDC_RX_ENDPOINT       3
      #define CDC_TX_ENDPOINT       3
      #define CDC_ACM_SIZE          16
      #define CDC_RX_SIZE_480       512
      #define CDC_TX_SIZE_480       512
      #define CDC_RX_SIZE_12        64
      #define CDC_TX_SIZE_12        64
      
      #define MTP_INTERFACE   2 // MTP Disk
      #define MTP_TX_ENDPOINT 4
      #define MTP_TX_SIZE_12  64
      #define MTP_TX_SIZE_480 512
      #define MTP_RX_ENDPOINT 4
      #define MTP_RX_SIZE_12  64
      #define MTP_RX_SIZE_480 512
      #define MTP_EVENT_ENDPOINT  5
      #define MTP_EVENT_SIZE  32
      #define MTP_EVENT_INTERVAL_12 10  // 10 = 10 ms
      #define MTP_EVENT_INTERVAL_480 7  // 7 = 8 ms
    
      #define ENDPOINT2_CONFIG  ENDPOINT_RECEIVE_UNUSED + ENDPOINT_TRANSMIT_INTERRUPT
      #define ENDPOINT3_CONFIG  ENDPOINT_RECEIVE_BULK + ENDPOINT_TRANSMIT_BULK
      #define ENDPOINT4_CONFIG  ENDPOINT_RECEIVE_BULK + ENDPOINT_TRANSMIT_BULK
      #define ENDPOINT5_CONFIG  ENDPOINT_RECEIVE_UNUSED + ENDPOINT_TRANSMIT_INTERRUPT
    But as you said it should not hold up Beta1...

  13. #563
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    26,398
    I added some comments in the 3 storage examples about the begin() function being a temporary API.

    Planning to package up 1.57-beta1 in a couple hours....

  14. #564
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,019
    Cool ... Looking forward to 1.57b1 and getting back to this. Keeping up was a chore with variety of peripheral adds ...

    Seems I found the code in progress ...

  15. #565
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    8,236
    Quote Originally Posted by PaulStoffregen View Post
    I added some comments in the 3 storage examples about the begin() function being a temporary API.

    Planning to package up 1.57-beta1 in a couple hours....
    Very cool Paul - looking forward to it.

  16. #566
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,451
    Quote Originally Posted by KurtE View Post

    If instead of current 0x476 which was the everything PID
    We could use 0x4D2 I am pretty sure your udev rules would not have to change.

    like:
    Code:
    #elif defined(USB_MTPDISK_SERIAL)
      #define VENDOR_ID             0x16C0
      #define PRODUCT_ID            0x04D2  // New MTP + Serial
      #define RAWHID_USAGE_PAGE 0xFFAB  // recommended: 0xFF00 to 0xFFFF
      #define RAWHID_USAGE    0x0200  // recommended: 0x0100 to 0xFFFF
      #define DEVICE_CLASS    0xEF
      #define DEVICE_SUBCLASS 0x02
      #define DEVICE_PROTOCOL 0x01
    ....
    Not sure if the RAWHID_* and DEVICE_ * are needed and what they are doing
    they com from USB_EVERYTHING

    they are used in part for
    USB_SERIAL_HID (DEVICE_*)
    USB_RAW_HID (RAWHID_*)
    but
    USB_MIDI4,16 ,USB_FLIGHTSIM_JOYSTICK,USB_MIDI16_AUDIO_SERIAL use also BCD_DEVICE

    We can easily use 0x04D2 (I used a fake PID initially), but then we must convince Paul to add ii to the Teensy loader environment (I guess it is Teensy_reboot.exe) so that we can download without button press.

  17. #567
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,019
    TD 1.57b1 available - if IDE 2.0 tested/used check the build dir the IDE uses. After 1.57b1 installed IDE 1.8.19 still used %appdata%\...\1.56.1 install, not where installed in the IDE unZip folder.
    > noted on Beta 1.57b1 announce

    >> Found T_4.1 makefiles.ino that was creating uniform test files to verify.
    SD card from Feb sitting in that T_4.1 is oddly full and large from a world of stuff with 107K files:
    Code:
    Total 107,999 files of Size 1,088,487,493 Bytes
    Bytes Used: 28,699,394,048, Bytes Total:128,144,375,808
    > not exactly sure why the 27 extra GB between Size and Used, maybe a var overflow - folders and small files and large block alloc size?
    Code:
    Windows shows:
    107,991 files and 12,724 folders
    12.4GB Size and 24.6GB used on disk
    So only 8 files diff - some hidden Windows may not count.

    And doing the 'v' verify - built without MTP so just doing 'Serial' with MTP #ifdef out the code does a Teensy Restart - so need to discover that...

    Is there working MTP sketch in the install? Need to find the working changes for what was there before ...
    > total fail now without : #include <MTP_Teensy.h>

  18. #568
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    10,547
    Good morning @defragster -

    Not sure what you are asking? If you are asking about sizes of files versus bytes used...

    I am guessing it all depends on the cluster size of your storage. So for example if you have 10 files that 3 bytes long,

    that adds up to 30 bytes, but if your cluster size is 32K, those 10 files take up 320K on disk...

    I don't believe MTP_Teensy is included in this beta release. Many of the sketches in the current library may have issues working if they deal with MSC.
    Many of these you can turn this off, but the MSC interfaces are evolving right now. And as such Paul noted that within a few examples.

    @mjs513 and myself have updated a couple of them to semi-function with current version. Like right now my CircuitPython test code only handles the first partition
    of a disk plugged in. Why the partition code should soon be reintroduced in some form, where the disk code or like will do the enumeration and claim a partition FileSystem object for them...

    And likewise the Filesystem object does not have any method on it, to be released... again will be fixed.

  19. #569
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    16,019
    Quote Originally Posted by KurtE View Post
    Good morning @defragster -

    Not sure what you are asking? If you are asking about sizes of files versus bytes used...

    I am guessing it all depends on the cluster size of your storage. So for example if you have 10 files that 3 bytes long,

    that adds up to 30 bytes, but if your cluster size is 32K, those 10 files take up 320K on disk...

    I don't believe MTP_Teensy is included in this beta release. Many of the sketches in the current library may have issues working if they deal with MSC.
    Many of these you can turn this off, but the MSC interfaces are evolving right now. And as such Paul noted that within a few examples.

    @mjs513 and myself have updated a couple of them to semi-function with current version. Like right now my CircuitPython test code only handles the first partition
    of a disk plugged in. Why the partition code should soon be reintroduced in some form, where the disk code or like will do the enumeration and claim a partition FileSystem object for them...

    And likewise the Filesystem object does not have any method on it, to be released... again will be fixed.
    Good morning @KurtE, ...
    Only question was what is needed to get back to MTP current in sync with 1.57b1 ... wrongly assumed it was merged.
    > i.e. current examples for MTP and MSC, and any external sources.

    File size issue was just noted because the SD card is comically large and an issue HERE to fix.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •