CRC32 error with buffer size>= 64k

Uwe

Active member
Hello!

I think the CRC32 library has a bug if the buffer size containing the data is 64k or bigger.

I wrote a little function in java running on a Linux PC calculating the CRC32 of a dummy data field(containing all 'A'):

Code:
  private static void testCrc() {
    int bufSize = 1024 * 64;
    byte[] buffer = new byte[bufSize];
    CRC32 crc = new CRC32();
    for (int i = 0; i < bufSize; i++) {
      buffer[i] = 65;
    }
    crc.update(buffer, 0, bufSize);
    long crcVal = crc.getValue();
    System.out.println("CRC: " + crcVal);
  }

The result of this routine is CRC: 2694514304

The same function on a Teensy 4.1:

Code:
FastCRC32 CRC32;
void testCrc(){
  int bufSize = 1024 * 64;
  uint8_t buffer[bufSize];
  for (int i = 0; i < bufSize; i++) {
    buffer[i] = 65;
  }

  uint32_t crc = CRC32.crc32(&buffer[0], bufSize);

  Serial.print("CRC: ");
  Serial.println(crc);
}

shows CRC: 0

If I reduce the buffer size by 1 (bufSize = 1024 * 64 - 1) Java and the Teensy both show the same result ("CRC: 2660141191").

If I raise the bufferSize above 64k(e.g. +1) Java is calculating 1052870317 whereas my Teensy calculates 3554254475.

Did I miss something in the documentation? Is the Teensy CRC32 limited to 64k-1 or is it a bug?

Uwe
 
Oh, I think I can answer it by myself. I just looked into the source code of FastCRC. The length parameter for the buffer is of type uint16_t. So it looks like that FastCRC is limited to 64k-1.
 
My solution is to dissect my incoming data into portions <64k. To have a "real" solution FastCRC must be fixed. (At least FastCRC should return an error code if called with a buffersize >= 64k)
 
Note: I have idea if this will work or not, I did a quick change of his library, to convert the datalen from uint16_t to uint32_t and changed a few for loops I found that were indexing...

Other places I saw decremented the datalen and or created a pointer to after the data by adding datalen... These should work without changing...

I ran the benchmark and validation sketches on T3.2... Note I have never used this library so don't know if this will take care of all of it for everyone

And/Or there is downside for other platforms...

My Fork/Branch is up at: https://github.com/KurtE/FastCRC/tree/buflen_uint32_t

You might try it to see if it works ... If so can try to issue Pull Request...
 
...I am eagerly waiting for the (tested) pullrequest...
maybe do it in a way that considers AVR or other "CPU"s with low Flash/RAM?
 
Back
Top