Forum Rule: Always post complete source code & details to reproduce any issue!
Page 1 of 21 1 2 3 11 ... LastLast
Results 1 to 25 of 513

Thread: Many TLAs: MTP MSC FS SD SDFat LittleFS UsbMSCFat to work with each other 8)

Hybrid View

  1. #1
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774

    Many TLAs: MTP MSC FS SD SDFat LittleFS UsbMSCFat to work with each other 8)

    Not sure if some of this should go on MTP thread(https://forum.pjrc.com/threads/43050...-Contribution(

    and/or the MSC thread: https://forum.pjrc.com/threads/55821...l=1#post270839

    or ???

    But for the fun of it, thought I would see how hard it would be to add one or more MSC devices to our list of MTP devices in the mtp-test sketch.

    There are probably many issues and the like, with several different File Systems:

    SD for SD stuff

    LittleFS for the chips and the like

    and currently now UsbMSCFat which is a modified version of SD (or SDFat...) stuff, maybe at some point we can combine?

    So far, I have just tried to add the include files and maybe define the start of a few objects:

    Code:
    #define USE_MSC_FAT 1
    ...
    //=============================================================================
    // MSC FAT classes
    //=============================================================================
    #if USE_MSC_FAT == 1
    #if defined(__IMXRT1062__) || defined(ARDUINO_TEENSY36)
    #include <mscFS.h>
    USBHost myusb;
    USBHub hub1(myusb);
    USBHub hub2(myusb);
    
    // start off with one controller. 
    msController msDrive1(myusb);
    
    
    #else 
    // Only those Teensy support USB
    #warning "Only Teensy 3.6 and 4.x support MSC"
    #endif
    #endif
    And the build now fails in loop()
    Code:
        char pathname[MAX_FILENAME_LEN];
    where MAX_FILENAME_LEN is not defined...

    Turns out just about every library defines some generic name like: MAX_FILENAME_LEN

    And in this case, the mscFS.h file not only defines it:
    It then un-defines it:

    Code:
    // do not expose these defines in Arduino sketches or other libraries
    #undef MSCFAT_FILE
    #undef MSCFAT_BASE
    #undef MAX_FILENAME_LEN
    This one is probably easy to work around, as the only place this define is used in this directory is the actual header, so could change to something else.
    Or could change to something in the class itself, like an enum or a static const uint16_t MAX_FILENAME_LEN=256;

    But my real question here is should we just continue on the multiple different threads and/or create a new thread for Teensyduino Integration of hopefully getting
    all of these capabilities working with each other and hopefully integrated into some version of Teensyduino, and/or some common place to get instructions on
    how to get all of this to work?

  2. #2
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,704
    @KurtE

    My suggestion would be that if it is a specific library related issue it should be on the appropriate thread. For instance if LittleFS is formatting correctly or not recognizing a flash chip it would go on the LittleFS thread. If related to integration of the of all the filesystems probably could go on the integration thread.

    If I remember correctly @wwatson did have a mod to the SD library but was working on the FS linkage as well so probably a good place for that discussion would be here. My guess is that we should try and the LittleFS as the FS starting point which links back to SDFat?

    EDIT: Was curious so I tried mscFat: see post https://forum.pjrc.com/threads/55821...l=1#post270944
    Last edited by mjs513; 02-20-2021 at 10:21 PM.

  3. #3
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774
    @mjs513 - I totally agree that in many cases posting to a thread specific to a library for something specific to that library...

    But was wondering where for example would be a good place to post things that maybe effect multiple of these libraries.

    Example: Like maybe should we add some additional methods to the FS classes:

    For example: Should File class have methods to allow you to get and set modify date/time and/or create date and time?

    Should FS class have a virtual method for things like:
    bool quickFormat();
    bool lowLevelFormat(char progressChar=0);
    if lowLevelFormat, should there be a callback of some type to allow your own way to show progress>

    Should there be a method to get logical (or desired) write sizes (sector, cluster).

    Should there be some standard error reporting something? Like if opening file system is because there is no such device, versus device is not formatted, versus we don't support that FS...

    ...

    Note: changed the one define in MSC to MSC_MAX_FILENAME_LEN and now builds... Crashes at startup
    Now to start debugging!

  4. #4
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774
    Another issue came up today in the thread: https://forum.pjrc.com/threads/66335...l=1#post270956

    in there he is trying to enable the use of UNICODE file names:
    Ok, the example in MTP_t4 is working, but only if #define USE_EXFAT_UNICODE_NAMES 0 in ExFatConfig.h

    The code fails to build. He mentions in followup message:
    WMXZ library in storage.h seems to be char* focused, yes. I guess this would need to be changed.
    At the very least it is not compatible with the Unicode mode of SdFat.
    In the example, it gets an SDClass from the SD wrapper (in the Teensy libraries), and that also does not seem to support Unicode.
    And FS.h in Cores?

  5. #5
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,704
    Morning @KurtE

    Sorry I didn't get to this yesterday - still recovering a bit and had another distraction.

    Should FS class have a virtual method for things like:
    bool quickFormat();
    bool lowLevelFormat(char progressChar=0);
    if lowLevelFormat, should there be a callback of some type to allow your own way to show progress>

    Should there be a method to get logical (or desired) write sizes (sector, cluster).

    Should there be some standard error reporting something? Like if opening file system is because there is no such device, versus device is not formatted, versus we don't support that FS...
    I agree with 100%. For the things that you described those are the things that should/could be discussed on the integration thread. Especially going to come more into play as expanding usage via MSC. This was apparent when I tried the SSD drive. It actually has 2 partitions from when we were testing MSC, it showed up in one of the examples and how to get what we had with MSC into SDFat/FS etc. Same thing with error codes that BG has in the lib.

  6. #6
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,704
    Just including this for reference: https://forum.pjrc.com/threads/55821...l=1#post271015 on using partitions.

  7. #7
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    599
    @All - Ok, I have discovered a an unneeded delay in MSC that affects the time it takes for initializing a USB drive. In MassStorageDriver.cpp line #200 is:
    Code:
    	msReset(0); // Assume bNumInterfaces = 1 for now.
    //	delay(500);
    	maxLUN = msGetMaxLun(0); // Assume bNumInterfaces = 1 for now
    It is not necessary. Also, I found this comment in hathatch's tinyusb driver:
    Code:
    // MSC interface Reset (not used now)
    So I commented that out and found no ill effects when tested with my USB drives.

    Using 'volumeName.ino' I tested with and without the delay after 'msReset(0)'.

    Results with the delay:

    Code:
    Waiting up to 5 seconds for USB drive 1
    ^^^^Waiting up to 5 seconds for USB drive 2
    ^^^^^~Initialize USB drive...^^^^USB drive 1 is present.
    Without the delay:
    Code:
    Waiting up to 5 seconds for USB drive 1
    ^^^^Waiting up to 5 seconds for USB drive 2
    ^^^^^~Initialize USB drive...USB drive 1 is present.
    Commenting out 'msReset(0)' did not change anything else that I could detect so far.

    I feel the need, the need for speed

  8. #8
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774
    Sounds good!

    Will take in. Right now I am trying to make a version of the fsFat which is partition friendly.
    Running into a few compile issues. Will let everyone know if I have any luck.

  9. #9
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774
    @all, again not sure if best to mention here or in the MSC thread...

    But I just pushed up a new Branch of the UsbMscFat library... https://github.com/KurtE/UsbMscFat/tree/PFsLib

    Where I completely duplicated the SdFat/FsLib sub-library where the only real difference is that in the FsVolume class
    The begin method added the two same optional parameters that are in FatLib and ExFatLib:
    That is: bool begin(BlockDevice* blockDev);
    is now: bool begin(BlockDevice* dev, bool setCwv = true, uint8_t part = 1);

    And the implementation, went from:
    Code:
    bool FsVolume::begin(BlockDevice* blockDev) {
      Serial.printf("FsVolume::begin(%x)\n", (uint32_t)blockDev);
      m_blockDev = blockDev;
      m_fVol = nullptr;
      m_xVol = new (m_volMem) ExFatVolume;
      if (m_xVol && m_xVol->begin(m_blockDev, false)) {
        goto done;
      }
      m_xVol = nullptr;
      m_fVol = new (m_volMem) FatVolume;
      if (m_fVol && m_fVol->begin(m_blockDev, false)) {
        goto done;
      }
      m_cwv = nullptr;
      m_fVol = nullptr;
      return false;
    
     done:
      m_cwv = this;
      return true;
    }
    To:
    Code:
    bool PFsVolume::begin(BlockDevice* blockDev, bool setCwv, uint8_t part) {
    //  Serial.printf("PFsVolume::begin(%x)\n", (uint32_t)blockDev);
      m_blockDev = blockDev;
      m_fVol = nullptr;
      m_xVol = new (m_volMem) ExFatVolume;
      if (m_xVol && m_xVol->begin(m_blockDev, setCwv, part)) {
        goto done;
      }
      m_xVol = nullptr;
      m_fVol = new (m_volMem) FatVolume;
      if (m_fVol && m_fVol->begin(m_blockDev, setCwv, part)) {
        goto done;
      }
      m_cwv = nullptr;
      m_fVol = nullptr;
      return false;
    
     done:
      m_cwv = this;
      return true;
    }
    The only other things that changed where the names of files, classes, defines...

    I then reworked the VolumeName.ino sketch to use it. I also removed all of the specific get volume label functions for 16 bit 32 bit and ExFat and have it all in my now called getPartitionVolumeLabel, which if I keep the
    classes mentioned here. it all uses my new class and I personally think it is a lot cleaner... Note: I also have the SDCard code calling through this as well.
    As I now pass in the reference to one of the PfsVolume objects...


    Code:
    bool getPartitionVolumeLabel(PFsVolume &partVol, uint8_t *pszVolName, uint16_t cb) {
      uint8_t buf[512];
      if (!pszVolName || (cb < 12)) return false; // don't want to deal with it
    
      PFsFile root;
      if (!root.openRoot(&partVol)) return false;
      root.read(buf, 32);
      //print_hexbytes(buf, 32);
    
      switch (partVol.fatType())
      {
        case FAT_TYPE_FAT12:
        case FAT_TYPE_FAT16:
        case FAT_TYPE_FAT32:
          {
            DirFat_t *dir;
            dir = reinterpret_cast<DirFat_t*>(buf);
            if ((dir->attributes & 0x08) == 0) return false; // not a directory...
            size_t i;
            for (i = 0; i < 11; i++) {
              pszVolName[i]  = dir->name[i];
            }
            while ((i > 0) && (pszVolName[i - 1] == ' ')) i--; // trim off trailing blanks
            pszVolName[i] = 0;
          }
          break;
        case FAT_TYPE_EXFAT:
          {
            DirLabel_t *dir;
            dir = reinterpret_cast<DirLabel_t*>(buf);
            if (dir->type != EXFAT_TYPE_LABEL) return false; // not a label? 
            size_t i;
            for (i = 0; i < dir->labelLength; i++) {
              pszVolName[i] = dir->unicode[2 * i];
            }
            pszVolName[i] = 0;
          }
          break;
        } 
      return true;
    }
    Code:
    // Function to handle one MS Drive...
    void procesMSDrive(uint8_t drive_number, msController &msDrive, UsbFs &msc)
    {
      Serial.printf("Initialize USB drive...");
      cmsReport = 0;
      if (!msc.begin(&msDrive)) {
        Serial.println("");
        msc.errorPrint(&Serial);
        Serial.printf("initialization drive %u failed.\n", drive_number);
      } else {
        Serial.printf("USB drive %u is present.\n", drive_number);
      }
      cmsReport = -1;
    
      mbrDmp( &msc );
      for (uint8_t i = 1; i < 5; i++) {
        PFsVolume partVol;
        uint8_t volName[32];
        if (!partVol.begin(msc.usbDrive(), true, i)) continue; // not a valid volume.
        partVol.chvol();
    
        switch (partVol.fatType())
        {
          case FAT_TYPE_FAT12: Serial.print("\n>> Fat12: "); break;
          case FAT_TYPE_FAT16: Serial.print("\n>> Fat16: "); break;
          case FAT_TYPE_FAT32: Serial.print("\n>> Fat32: "); break;
          case FAT_TYPE_EXFAT: Serial.print("\n>> ExFat: "); break;
        }
        if (getPartitionVolumeLabel(partVol, volName, sizeof(volName))) {
          Serial.printf("Volume name:(%s)", volName);
        }
        elapsedMicros em_sizes = 0;
        uint64_t used_size =  (uint64_t)(partVol.clusterCount() - partVol.freeClusterCount())
                          * (uint64_t)partVol.bytesPerCluster();
        uint64_t total_size = (uint64_t)partVol.clusterCount() * (uint64_t)partVol.bytesPerCluster();
        Serial.printf(" Partition Total Size:%llu Used:%llu time us: %u\n", total_size, used_size, (uint32_t)em_sizes);
    
        partVol.ls();
      }
    }
    The function processing each SD Drive is pretty simple now as well:
    Code:
      Serial.printf("\nInitialize SD card...");
    
      if (!sd.begin(SD_CONFIG)) {
        Serial.println("initialization failed.\n");
      } else {
        Serial.println("SD card is present.\n");
        PFsVolume partVol;
        if (!partVol.begin(sd.card(), true, 1)) Serial.println("SD Did not open partition...");
        switch (partVol.fatType())
        {
          case FAT_TYPE_FAT12: Serial.print("\n>> Fat12: "); break;
          case FAT_TYPE_FAT16: Serial.print("\n>> Fat16: "); break;
          case FAT_TYPE_FAT32: Serial.print("\n>> Fat32: "); break;
          case FAT_TYPE_EXFAT: Serial.print("\n>> ExFat: "); break;
        }
        uint8_t volName[32];
        if (getPartitionVolumeLabel(partVol, volName, sizeof(volName))) {
          Serial.printf("Volume name:(%s)\n", volName);
        }
    A test run:
    Code:
    Waiting up to 5 seconds for USB drive 1
    ^^Waiting up to 5 seconds for USB drive 2
    ^^^^^^^~~~~:(Initialize USB drive...^^^^^^^FsVolume::begin(20001f70)
    USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x4,0x1,0x4,0xB,0xFE,0xC2,0xFF,2048,8192000
    FAT16:	2,0,0xE,0x51,0xFE,0xE,0x98,0x98,0x80,8194048,2097152
    exFAT:	3,0,0x98,0x99,0x80,0x7,0xEE,0xDC,0xD2,10291200,5435392
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    >> Fat32: Volume name:(VOLFAT32) Partition Total Size:4186095616 Used:65425408 time us: 6074373
    DSC03357.JPG
    example.txt
    OnFat32.txt
    IMG_0277.jpg
    IMG_1240.jpg
    IMG_1238.jpg
    IMG_1236.jpg
    IMG_1235.jpg
    IMG_1229.jpg
    teensy41_BP.jpg
    teensy41_2aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.jpg
    T4-Card-Front.jpg
    IMG_1176.jpg
    IMG_1167.jpg
    lynxmotion.png
    visualMicro.jpg
    IMG_1162.jpg
    IMG_1163.jpg
    IMG_1164.jpg
    IMG_0267.jpg
    IMG_0266.jpg
    IMG_0264.jpg
    IMG_1154.jpg
    T4.1-Cardlike.jpg
    T4-Cardlike.jpg
    IMG_1304.jpg
    IMG_1301.jpg
    IMG_1303.jpg
    IMG_1302.jpg
    IMG_1300.jpg
    IMG_0287.jpg
    IMG_0283.jpg
    IMG_1267.jpg
    IMG_1260.jpg
    IMG_1258.jpg
    IMG_1259.jpg
    IMG_1255.jpg
    IMG_1254.jpg
    IMG_0278.jpg
    Ausi Pictures/
    P3190014.JPG
    P3190015.JPG
    P3190016.JPG
    P3100001.JPG
    P3100003.JPG
    P3150004.JPG
    P3150005.JPG
    P3180006.JPG
    P3190007.JPG
    P3190008.JPG
    P3190009.JPG
    P3190010.JPG
    P3190011.JPG
    P3190012.JPG
    P3190013.JPG
    DSC03521.JPG
    DSC03522.JPG
    DSC03518.JPG
    DSC03519.JPG
    DSC03520.JPG
    
    >> Fat16: Volume name:(VOLFAT16) Partition Total Size:1073446912 Used:11714560 time us: 193125
    T4-Cardlike.jpg
    OnFat16.txt
    DSC03357.JPG
    Inside-Ps2.jpg
    Arduino-close-up.jpg
    Birds.jpg
    Encoder-plus-serial-overvie.jpg
    Oscilloscope.bmp
    Encoder-logic-scan.jpg
    tri-track-back.jpg
    DSC03361.JPG
    Serout-and-TASerial-at-1152.jpg
    Chr3-rotate-RJoy-LR.jpg
    DSC03358.JPG
    
    >> ExFat: Volume name:(VolEXFAT) Partition Total Size:2782461952 Used:232030208 time us: 17002
    T4.1-Cardlike.jpg
    OnExFat.txt
    Hitec-laser-6-receiver.jpg
    Arduino-mega-Brat.jpg
    Arduino-mega-with-PS2.jpg
    Add-pull-up-to-PS2-receiver.jpg
    Inside-Ps2.jpg
    Arduino-close-up.jpg
    Two-Arduino-Pros.jpg
    Arduino-SSC32-and-Pro40.jpg
    Birds.jpg
    Encoder-plus-serial-overvie.jpg
    Encoders-plus-serial-to-and.jpg
    Encoder-on-USB-Oscilloscope.jpg
    Oscilloscope.bmp
    Encoder-logic-scan.jpg
    nano-serout.jpg
    Tri-track-front.jpg
    tri-track-back.jpg
    DSC03361.JPG
    DSC03360.JPG
    Minor-Nit-on-startup.jpg
    Serout-and-TASerial-at-1152.jpg
    Chr3-rotate-RJoy-LR.jpg
    Arduino-DIY-remote.jpg
    BB-DUINO-Shield-BRD-Bottom.jpg
    bb-bap64-xbee.jpg
    joysticks.jpg
    Botboarduino-top.jpg
    Botboarduino-bottom.jpg
    DIY-Remote-with-updates.jpg
    RC-input-timings.jpg
    BB-Duino-Jr-SCH.jpg
    Seeeduino-Mega-with-XBee-Sh.jpg
    Lynxmotion-Prop-Board-Back.jpg
    Lynxmotion-Prop-board-Front.jpg
    Lynxmotion-Propeller-Schema.jpg
    Lynxmotion-Propeller-Board.jpg
    Propeller-Multi-Controller-.jpg
    Ps2-with-cable.jpg
    Tri-track-with-roboclaw.jpg
    webbotlib-serial-closeup.jpg
    Webbotlib-serial.jpg
    belkin-play-serial-2.jpg
    Belkin-play-serial-1.jpg
    Seeduino-Mega.jpg
    Lantronix-board.jpg
    chr3-with-new-body2.jpg
    CHR3-with-new-body.jpg
    Trace-Nano-I2c-Read.jpg
    Trace-Nano-I2C-write.jpg
    CHR3-leg-body-up-battery-he.jpg
    CHR3-leg-body-on-ground.jpg
    Hitec-laser6-logic-scan.jpg
    Jake/
    
    Initialize SD card...FsVolume::begin(20003db0)
    SD card is present.
    
    
    >> ExFat: Volume name:(EXFat_Vol)
    done...
    Press any key to run again
    I ran into another interesting thing with an Ubuntu Install (Same type thumb drive):
    Code:
    Waiting up to 5 seconds for USB drive 2
    ^^^^^^^^^~~~~Initialize USB drive...^^^^^^^^^~FsVolume::begin(20001f70)
    
    Check USB drive format.
    initialization drive 1 failed.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    pt_#0:	1,80,0x0,0x1,0x0,0x0,0xAB,0xE0,0xFC,0,5619584
    pt_#239:	2,0,0xFE,0xFF,0xFF,0xEF,0xFE,0xFF,0xFF,1700,8000
    pt_#131:	3,0,0xCE,0x72,0x5D,0x83,0xF,0xFC,0xD3,5619712,10108928
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    Initialize SD card...FsVolume::begin(20003db0)
    SD card is present.
    
    
    >> ExFat: Volume name:(EXFat_Vol)
    done...
    Press any key to run again
    Note it does not like any of the partitions... But if plugged into PC the first partiation shows up as a FAT drive:
    Note the 80 in the Boot field...

    Need to experiment and find RPI SD Card and see what shows up.



    Let me know what you think. Maybe I will play with looking to see if the SD card has partitions as well.

    I will probably go ahead and issue a Pull request for it.

  10. #10
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,704
    @KurtE
    Just tried compiling for the T4.1 (VolumeName.ino) in the examples folder and got the following error:
    Code:
    C:\Users\Merli\AppData\Local\Temp\arduino_build_281257\libraries\SdFat\FsLib\FsNew.cpp.o: In function `operator new(unsigned int, unsigned long*)':
    F:\arduino-1.8.13-beta6\hardware\teensy\avr\libraries\SdFat\src\FsLib/FsNew.cpp:29: multiple definition of `operator new(unsigned int, unsigned long*)'
    C:\Users\Merli\AppData\Local\Temp\arduino_build_281257\libraries\UsbMscFat-PFsLib\PFsLib\PFsNew.cpp.o:D:\Users\Merli\Documents\Arduino\libraries\UsbMscFat-PFsLib\src\PFsLib/PFsNew.cpp:29: first defined here
    f:/arduino-1.8.13-beta6/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld.exe: Disabling relaxation: it will not work with multiple definitions

  11. #11
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    599
    @KurtE - Same here as @mjs513.

    Code:
    /home/wwatson/arduino-1.8.13/hardware/teensy/avr/libraries/SdFat/src/FsLib/FsNew.cpp:29: multiple definition of `operator new(unsigned int, unsigned long*)'
    /tmp/arduino_build_850402/libraries/UsbMscFat-PFsLib/PFsLib/PFsNew.cpp.o:/home/wwatson/Arduino/libraries/UsbMscFat-PFsLib/src/PFsLib/PFsNew.cpp:29: first defined here
    /home/wwatson/arduino-1.8.13/hardware/tools/arm/bin/../lib/gcc/arm-none-eabi/5.4.1/../../../../arm-none-eabi/bin/ld: Disabling relaxation: it will not work with multiple definitions
    collect2: error: ld returned 1 exit status
    Under Linux 18.04.

  12. #12
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    599
    @KurtE Was wondering why you were silent for so long This is awesome

    Will download and test. That is a lot of work in a short time.

    Thanks

  13. #13
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,704
    @KurtE
    Fantastic job.... think duplicating the sub-library was the best approach and then modifing it. Will download and give it a try.

  14. #14
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,280
    Same build issue here on prior code. Must be another moving part?

    I like the idea of having one thread for this combination effort for context - so here is fine without jumping threads or polluting the wrong one.

    Might even be nice to have a github_Combo with a subfolder of all the LIBS affected to aid in synchonized update ? Sort of like worked with git.LFSIntegrity so WIP updates could go in without banging into an official FORK?

  15. #15
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    599
    Quote Originally Posted by defragster View Post
    Same build issue here on prior code. Must be another moving part?

    I like the idea of having one thread for this combination effort for context - so here is fine without jumping threads or polluting the wrong one.

    Might even be nice to have a github_Combo with a subfolder of all the LIBS affected to aid in synchonized update ? Sort of like worked with git.LFSIntegrity so WIP updates could go in without banging into an official FORK?
    Me too, bouncing between threads at my age guaranties I will miss something. We are down one person at work so this week I might not be able play much

    Dinner and bed now...

  16. #16
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774
    Try syncing again. I thought I had disabled that new... Turned out I had disabled the one in SDFat library instead

  17. #17
    Senior Member wwatson's Avatar
    Join Date
    Aug 2017
    Posts
    599
    Quote Originally Posted by KurtE View Post
    Try syncing again. I thought I had disabled that new... Turned out I had disabled the one in SDFat library instead
    It's working Will do the PR.

  18. #18
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,280
    @All here : wwatson, mjs513, KurtE ...

    If I made the Unified multi Lib on github and gave all write access we wouldn't need PR's (or catching diff edits) - just fixes?

    Does that have validity - then this thread would have a single git source for the disparate elements.

  19. #19
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774
    Quote Originally Posted by defragster View Post
    @All here : wwatson, mjs513, KurtE ...

    If I made the Unified multi Lib on github and gave all write access we wouldn't need PR's (or catching diff edits) - just fixes?

    Does that have validity - then this thread would have a single git source for the disparate elements.
    Maybe... But then there is also MTP with @WMXZ and the like...

    Quick fyi - right after the merge,

    I updated the sketch again, such that you can dump the MBR for both USB devices as well as SD Cards... I pushed it up in same branch after the merge will do it again in new branch
    or new combined project...

    In case you wish to edit in (or get from previous branch)

    Code:
    bool mbrDmp(BlockDeviceInterface *blockDev) {
      MbrSector_t mbr;
      // bool valid = true;
      if (!blockDev->readSector(0, (uint8_t*)&mbr)) {
        Serial.print("\nread MBR failed.\n");
        //errorPrint();
        return false;
      }
      Serial.print("\nmsc # Partition Table\n");
      Serial.print("\tpart,boot,bgnCHS[3],type,endCHS[3],start,length\n");
      for (uint8_t ip = 1; ip < 5; ip++) {
        MbrPart_t *pt = &mbr.part[ip - 1];
        //    if ((pt->boot != 0 && pt->boot != 0X80) ||
        //        getLe32(pt->relativeSectors) > sdCardCapacity(&m_csd)) {
        //      valid = false;
        //    }
        switch (pt->type) {
          case 4:
          case 6:
          case 0xe:
            Serial.print("FAT16:\t");
            break;
          case 11:
          case 12:
            Serial.print("FAT32:\t");
            break;
          case 7:
            Serial.print("exFAT:\t");
            break;
          default:
            Serial.print("pt_#");
            Serial.print(pt->type);
            Serial.print(":\t");
            break;
        }
        Serial.print( int(ip)); Serial.print( ',');
        Serial.print(int(pt->boot), HEX); Serial.print( ',');
        for (int i = 0; i < 3; i++ ) {
          Serial.print("0x"); Serial.print(int(pt->beginCHS[i]), HEX); Serial.print( ',');
        }
        Serial.print("0x"); Serial.print(int(pt->type), HEX); Serial.print( ',');
        for (int i = 0; i < 3; i++ ) {
          Serial.print("0x"); Serial.print(int(pt->endCHS[i]), HEX); Serial.print( ',');
        }
        Serial.print(getLe32(pt->relativeSectors), DEC); Serial.print(',');
        Serial.println(getLe32(pt->totalSectors));
      }
      return true;
    }
    The call for USB Devices looks like: mbrDmp( msc.usbDrive() );

    and the one for SDCard looks like: mbrDmp(sd.card() );

    Next up actuallly to change the processMSDrive function,
    to take in the Blockdevice and then it can handle the whole thing...

  20. #20
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774
    @mjs513 Long Pauses are often the trying to compute the used or free space... As it walk billions and billions of ...

    Probably it for today. Probably tomorrow will look into MTP again with these changes.

    I am curious about how to get it to work on both PC and Ubuntu with the long long grab of information.

    Again thinking of two approaches:

    a) Lie at the start: and tell host that some bogus value... Or there 0xffffffff value. then after everything appears to settle, then get the actual values. and send event saying that the properties changed...
    b) Allow the Host to talk to MTP while we are still computing stuff, but do like I did when we were in the process of doing format and error out every message they send us with the status we are busy...

    Should be fun!

  21. #21
    Senior Member+ mjs513's Avatar
    Join Date
    Jul 2014
    Location
    New York
    Posts
    7,704
    Just re-synched and seems to be working. Put in the SD Card Adatpter with 3 partitions:
    Code:
    Waiting up to 5 seconds for USB drive 1
    ^Waiting up to 5 seconds for USB drive 2
    ^^^^^^^^~~~~:(Initialize USB drive...^^^^^^^^USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,80,0x0,0x1,0x10,0xC,0x3,0xE0,0xFF,2048,524288
    FAT32:	2,0,0xC2,0x23,0x20,0xC,0xFE,0xFF,0xFF,526336,16564224
    exFAT:	3,0,0xFE,0xFF,0xFF,0x7,0xFE,0xFF,0xFF,17092608,232642560
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    >> Fat32: Volume name:(system-boot)###FatPartition::freeClusterCount: FT:32 start:2080 todo:516192
        m_sectorsPerCluster:1
        m_clusterSectorMask:0
        m_sectorsPerClusterShift:0
        m_fatType:32
        m_rootDirEntryCount:0
        m_allocSearchStart:1
        m_sectorsPerFat:4033
        m_dataStartSector:10146
        m_fatStartSector:2080
        m_lastCluster:516191
        m_rootDirStart:2
        m_bytesPerSector:512
     Partition Total Size:264289280 Used:961024 time us: 2116123
    bench0.txt
    bench1.txt
    bench2.txt
    bench3.txt
    bench4.txt
    bench5.txt
    
    >> Fat32: Volume name:(NEW VOLUME1)###FatPartition::freeClusterCount: FT:32 start:526814 todo:2066434
        m_sectorsPerCluster:8
        m_clusterSectorMask:7
        m_sectorsPerClusterShift:3
        m_fatType:32
        m_rootDirEntryCount:0
        m_allocSearchStart:1
        m_sectorsPerFat:16145
        m_dataStartSector:559104
        m_fatStartSector:526814
        m_lastCluster:2066433
        m_rootDirStart:2
        m_bytesPerSector:512
     Partition Total Size:8464105472 Used:983040 time us: 8072748
    bench0.txt
    bench1.txt
    bench2.txt
    bench3.txt
    bench4.txt
    bench5.txt
    
    >> ExFat: Volume name:(New Volume2) Partition Total Size:119107747840 Used:7077888 time us: 111002
    CommandStation-EX-Arch-v1-0.pdf
    VSLAM_and_Navigation_System_of_Unmanned_Ground_Vehicle_Based_on_RGB-D_Camera.pdf
    
    Initialize SD card...initialization failed.
    Did seem like it took a long pause when accessing the 2nd and 3rd partition.

  22. #22
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,280
    Indeed if it can work include @all

    >> @All here : wwatson, mjs513, KurtE. WMXZ ...

    Lots of libs in :: MTP-MSC-FS-SD-SDFat-LittleFS-UsbMSCFat

    Other: USBHost@PJRC

  23. #23
    Senior Member+ KurtE's Avatar
    Join Date
    Jan 2014
    Posts
    9,774
    Just pushed up a new branch off of main which I synced back up: https://github.com/KurtE/UsbMscFat/tree/VolumeName_SD

    It now shows Volume information about SD Card as well:
    Code:
    ^^^^^^^^~FsVolume::begin(20001f30)
    
    Check USB drive format.
    initialization drive 1 failed.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    pt_#0:	1,80,0x0,0x1,0x0,0x0,0xAB,0xE0,0xFC,0,5619584
    pt_#239:	2,0,0xFE,0xFF,0xFF,0xEF,0xFE,0xFF,0xFF,1700,8000
    pt_#131:	3,0,0xCE,0x72,0x5D,0x83,0xF,0xFC,0xD3,5619712,10108928
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    Initialize SD card...FsVolume::begin(20003d70)
    SD card is present.
    
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    exFAT:	1,0,0x82,0x3,0x0,0x7,0xFE,0xFF,0xFF,8192,62543872
    pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    >> ExFat: Volume name:(EXFat_Vol) Partition Total Size:32017219584 Used:4587520 time us: 7271
    T4.1-Cardlike.jpg
    T4-Cardlike.jpg
    DSC03355.JPG
    test1.txt
    mtpindex.dat
    done...
    Press any key to run again
    Waiting up to 5 seconds for USB drive 2
    ^^^^^^^^^~~~~Initialize USB drive...^^^^FsVolume::begin(20001f30)
    
    Check USB drive format.
    initialization drive 1 failed.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    pt_#0:	1,80,0x0,0x1,0x0,0x0,0xAB,0xE0,0xFC,0,5619584
    pt_#239:	2,0,0xFE,0xFF,0xFF,0xEF,0xFE,0xFF,0xFF,1700,8000
    pt_#131:	3,0,0xCE,0x72,0x5D,0x83,0xF,0xFC,0xD3,5619712,10108928
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    Initialize SD card...FsVolume::begin(20003d70)
    SD card is present.
    
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x0,0x1,0x40,0xC,0x3,0xE0,0xFF,8192,524288
    pt_#131:	2,0,0x3,0xE0,0xFF,0x83,0x3,0xE0,0xFF,532480,16269312
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    >> Fat32: Volume name:(boot) Partition Total Size:264289280 Used:47383552 time us: 94778
    overlays/
    bcm2708-rpi-b-plus.dtb
    COPYING.linux
    LICENCE.broadcom
    issue.txt
    bcm2708-rpi-b-rev1.dtb
    bcm2708-rpi-b.dtb
    bcm2708-rpi-cm.dtb
    bcm2708-rpi-zero-w.dtb
    bcm2708-rpi-zero.dtb
    bcm2709-rpi-2-b.dtb
    bcm2710-rpi-2-b.dtb
    bcm2710-rpi-3-b-plus.dtb
    bcm2710-rpi-3-b.dtb
    bcm2710-rpi-cm3.dtb
    bcm2711-rpi-4-b.dtb
    bcm2711-rpi-400.dtb
    bcm2711-rpi-cm4.dtb
    bootcode.bin
    cmdline.txt
    config.txt
    fixup.dat
    fixup4.dat
    fixup4cd.dat
    fixup4db.dat
    fixup4x.dat
    fixup_cd.dat
    fixup_db.dat
    fixup_x.dat
    kernel.img
    kernel7.img
    kernel7l.img
    kernel8.img
    start.elf
    start4.elf
    start4cd.elf
    start4db.elf
    start4x.elf
    start_cd.elf
    start_db.elf
    start_x.elf
    done...
    Press any key to run again
    Waiting up to 5 seconds for USB drive 1
    ^^Waiting up to 5 seconds for USB drive 2
    ^^^^^^^~~~~:(Initialize USB drive...^^^^^^^^^~FsVolume::begin(20001f30)
    
    Check USB drive format.
    initialization drive 1 failed.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    pt_#0:	1,80,0x0,0x1,0x0,0x0,0xAB,0xE0,0xFC,0,5619584
    pt_#239:	2,0,0xFE,0xFF,0xFF,0xEF,0xFE,0xFF,0xFF,1700,8000
    pt_#131:	3,0,0xCE,0x72,0x5D,0x83,0xF,0xFC,0xD3,5619712,10108928
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    Initialize SD card...FsVolume::begin(20003d70)
    SD card is present.
    
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x0,0x1,0x40,0xC,0x3,0xE0,0xFF,8192,524288
    pt_#131:	2,0,0x3,0xE0,0xFF,0x83,0x3,0xE0,0xFF,532480,16269312
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    >> Fat32: Volume name:(boot) Partition Total Size:264289280 Used:47383552 time us: 94772
    overlays/
    bcm2708-rpi-b-plus.dtb
    COPYING.linux
    LICENCE.broadcom
    issue.txt
    bcm2708-rpi-b-rev1.dtb
    bcm2708-rpi-b.dtb
    bcm2708-rpi-cm.dtb
    bcm2708-rpi-zero-w.dtb
    bcm2708-rpi-zero.dtb
    bcm2709-rpi-2-b.dtb
    bcm2710-rpi-2-b.dtb
    bcm2710-rpi-3-b-plus.dtb
    bcm2710-rpi-3-b.dtb
    bcm2710-rpi-cm3.dtb
    bcm2711-rpi-4-b.dtb
    bcm2711-rpi-400.dtb
    bcm2711-rpi-cm4.dtb
    bootcode.bin
    cmdline.txt
    config.txt
    fixup.dat
    fixup4.dat
    fixup4cd.dat
    fixup4db.dat
    fixup4x.dat
    fixup_cd.dat
    fixup_db.dat
    fixup_x.dat
    kernel.img
    kernel7.img
    kernel7l.img
    kernel8.img
    start.elf
    start4.elf
    start4cd.elf
    start4db.elf
    start4x.elf
    start_cd.elf
    start_db.elf
    start_x.elf
    done...
    Press any key to run again

  24. #24
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,280
    IGNORE THIS POST - The external HUB/HDD was unplugged

    The SD card on the PC looks like this with 4 Parts:
    Click image for larger version. 

Name:	SD4PartWin.png 
Views:	48 
Size:	35.9 KB 
ID:	23878
    Last edited by defragster; 03-01-2021 at 05:00 AM.

  25. #25
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    15,280
    UPDATING to DELETE PRIOR POST - external drive had POWER OFF - it FELL once - changed Teensy setup on desk and put rubber feet on External HUB/DRIVE.

    Got the updated p#23 UsbMsc. Using the VolumeName.ino sketch ( older p#522 sketch doesn't show? )

    This shows SD card info - but is wholly missing my USB HDD and FLASH? OPPS - that drive was unplugged!

    The SD card on the PC looks like this with 4 Parts:
    Click image for larger version. 

Name:	SD4PartWin.png 
Views:	48 
Size:	35.9 KB 
ID:	23878

    After HUB internal 120GB and 128GB Flash - The SD CARD output catches 4 partitions - but has wrong TYPE for ExFat 4th Part - and does not do a DIR on it?
    -- > It shows Nothing for the 32GB Flash drive, but works for the twin Part 64GB RECOVERY DRIVE - BOTTOM : Both 32GB Parts look okay.
    >> IT IS TYPE #15 ??? - adding that to accepted list does not work?
    -->> Put the SD card back into PC and :: GOT PC FAULT - Green Screen Fault - as seen during creating each of those 4 SD partitions ???
    - Not sure where Dir gets done from - is it Root only?
    Code:
    Waiting up to 5 seconds for USB drive 1
    ^^^^Waiting up to 5 seconds for USB drive 2
    Initialize USB drive...^^^^^^^^USB drive 1 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    exFAT:	1,0,0x20,0x21,0x0,0x7,0xFE,0xFF,0xFF,2048,234436608
    pt_#0:	2,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    >> ExFat: Volume name:(120GB_Ext) Partition Total Size:120026300416 Used:197394432 time us: 84126
    A_00001.dat
    A_00002.dat
    A_00003.dat
    A_00004.dat
    A_00005.dat
    A_00006.dat
    A_00007.dat
    Initialize USB drive...^^^^^^^
    Check USB drive format.
    initialization drive 2 failed.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT16:	1,0,0x20,0x21,0x0,0xE,0xE8,0x27,0x1,2048,28672
    exFAT:	2,0,0xA,0x9,0x2,0x7,0xFE,0xFF,0xFF,32768,242450176
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    >> ExFat:  Partition Total Size:124117712896 Used:1179648 time us: 244251
    ExFat.txt
    ExFatFldr/
    
    Initialize SD card...SD card is present.
    
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,0,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,16777216
    exFAT:	2,0,0xFE,0xFF,0xFF,0x7,0xFE,0xFF,0xFF,16779264,33554432
    FAT32:	3,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,50333696,67108864
    exFAT:	4,0,0xFE,0xFF,0xFF,0xF,0xFE,0xFF,0xFF,117442560,126834688  //  THIS PART SHOWED as pt_#15 ???
    
    >> Fat32: Volume name:(SD1_8GB_AD)###FatPartition::freeClusterCount: FT:32 start:10262 todo:1047042
        m_sectorsPerCluster:16
        m_clusterSectorMask:15
        m_sectorsPerClusterShift:4
        m_fatType:32
        m_rootDirEntryCount:0
        m_allocSearchStart:1
        m_sectorsPerFat:8181
        m_dataStartSector:26624
        m_fatStartSector:10262
        m_lastCluster:1047041
        m_rootDirStart:2
        m_bytesPerSector:512
     Partition Total Size:8577351680 Used:40960 time us: 187706
    SD1_8GB_AD.txt
    SD1_8GB_AD/
    
    >> ExFat: Volume name:(SD2_16GB_AD) Partition Total Size:17175674880 Used:262144 time us: 3727
    SD2_16GB_AD/
    SD2_16GB_AD.txt
    
    >> Fat32: Volume name:(SD3_32GB_AD)###FatPartition::freeClusterCount: FT:32 start:50341892 todo:1048194
        m_sectorsPerCluster:64
        m_clusterSectorMask:63
        m_sectorsPerClusterShift:6
        m_fatType:32
        m_rootDirEntryCount:0
        m_allocSearchStart:1
        m_sectorsPerFat:8190
        m_dataStartSector:50358272
        m_fatStartSector:50341892
        m_lastCluster:1048193
        m_rootDirStart:2
        m_bytesPerSector:512
     Partition Total Size:34347155456 Used:163840 time us: 187756
    SD3_32GB_AD/
    SD3_32GB_AD.txt
    // No Info or Dir Data for 4th Part that is ExFat 60 GB
    done...
    Press any key to run again
    Here is Restarting with Revovery 64GB drive in the system:
    Code:
    ...
    
    Initialize USB drive...^^^^USB drive 2 is present.
    
    msc # Partition Table
    	part,boot,bgnCHS[3],type,endCHS[3],start,length
    FAT32:	1,80,0x20,0x21,0x0,0xC,0xFE,0xFF,0xFF,2048,67108864
    FAT32:	2,0,0xFE,0xFF,0xFF,0xC,0xFE,0xFF,0xFF,67110912,54128640
    pt_#0:	3,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    pt_#0:	4,0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0,0
    
    >> Fat32: Volume name:(B )###FatPartition::freeClusterCount: FT:32 start:10244 todo:1048194
        m_sectorsPerCluster:64
        m_clusterSectorMask:63
        m_sectorsPerClusterShift:6
        m_fatType:32
        m_rootDirEntryCount:0
        m_allocSearchStart:1
        m_sectorsPerFat:8190
        m_dataStartSector:26624
        m_fatStartSector:10244
        m_lastCluster:1048193
        m_rootDirStart:2
        m_bytesPerSector:512
     Partition Total Size:34347155456 Used:28836429824 time us: 3071623
    efi/
    bootmgr
    boot/
    bootmgr.efi
    sources/
    reagent.xml
    MBAMJan_2019.PNG
    MBAM_TWO.PNG
    Mejii_MBAM.txt
    HWMonitor_x64.exe
    hwmonitorw.ini
    MyPhoneExplorer/
    arduino_b1.9_B27/
    TyComm/
    tCode/
    UserBack.bat
    smonteith/
    casa/
    irs2017/
    TUN-320 Win 10 Driver/
    
    >> Fat32: Volume name:(64G_PT#2)###FatPartition::freeClusterCount: FT:32 start:67117264 todo:1690498
        m_sectorsPerCluster:32
        m_clusterSectorMask:31
        m_sectorsPerClusterShift:5
        m_fatType:32
        m_rootDirEntryCount:0
        m_allocSearchStart:1
        m_sectorsPerFat:13208
        m_dataStartSector:67143680
        m_fatStartSector:67117264
        m_lastCluster:1690497
        m_rootDirStart:2
        m_bytesPerSector:512
     Partition Total Size:27697086464 Used:65536 time us: 4953373
    FPartTwo.txt
    
    ...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •