Different CRC when using Platformio and Arduino IDE

[samm_flynn]

I am using a teensy 4.1

I am trying to calculate CRC32 of a large float array, when I upload the code using arduino IDE, the result matches with the CRC32 calculated in python

how ever, when I upload the same code using platformio, the CRC is different.

"""
python  : CRC32 of float array: 0x6CCB2E56
arduino : CRC32 of float array: 0x6CCB2E56
pio     : CRC32 of float array: 0x6E3BCDC6
"""

I also get a warning -

C:/Users/___/OneDrive/Documents/PlatformIO/Projects/250222-170929-teensy41/src/FastCRC_CRC32.ino: In function 'void loop()':

C:/Users/___/OneDrive/Documents/PlatformIO/Projects/250222-170929-teensy41/src/FastCRC_CRC32.ino:33:100: warning: conversion from 'unsigned int' to 'uint16_t' {aka 'short unsigned int'} changes value from '511872' to '53120' [-Woverflow]

   33 |       Serial.printf("CRC32 of float array: 0x%08X\n", CRC32.crc32(reinterpret_cast<uint8_t*>(arr), sizeof(arr)));

arduino/platformio code -

#include <Arduino.h>
#include <FastCRC.h>

#define N 21328
#define M 6
DMAMEM float arr[N][M];  // Large float array
bool hasRun = false;  // Flag to ensure it runs only once per connection
void setup()
{
  Serial.begin(115200);  // Initialize serial communication
  for (int i = 0; i < N; i++)
  {
    for (int j = 0; j < M; j++)
    {
      arr[i][j] = static_cast<float>(i * M + j);  // Matches np.arange()
    }
  }
}
void loop()
{
  if (Serial.dtr())  // When a Python script opens the serial port
  {
    if (!hasRun)
    {
      FastCRC32 CRC32;
      hasRun = true;  // Set flag to prevent multiple runs during connection
      Serial.printf("CRC32 of float array: 0x%08X\n", CRC32.crc32(reinterpret_cast<uint8_t*>(arr), sizeof(arr)));
    }
  }
  else
  {
    hasRun = false;  // Reset when Python disconnects
  }
}

Python code -

import binascii
import numpy as np
import serial

SERIAL_PORT = "COM8"
BAUD_RATE = 4608000


def compute_crc32(data: bytes) -> int:
    return binascii.crc32(data) & 0xFFFFFFFF


# Example Usage
if __name__ == "__main__":
    # Example 1: Compute CRC32 of a float array (N x M)
    N, M = 21328, 6  # Same as C++
    float_arr = np.arange(N * M, dtype=np.float32).reshape(N, M)
    crc_float_array = compute_crc32(float_arr.tobytes())
    print(f"python : CRC32 of float array: 0x{crc_float_array:08X}")
    with serial.Serial(SERIAL_PORT, BAUD_RATE, timeout=0.1) as teensy:
        recv = teensy.readline()
        print(f"teensy : {recv.decode()}")
"""
python  : CRC32 of float array: 0x6CCB2E56
arduino : CRC32 of float array: 0x6CCB2E56
pio     : CRC32 of float array: 0x6E3BCDC6
"""

 

[joepasquariello]

I don't use PIO, so I'm guessing, but your DMAMEM array leaves 0 bytes free in RAM2, and perhaps there is something different about the linker script in PIO that results in some of those bytes being used. Have you tried testing with a smaller array? If that works, then you may simply be making the array larger than the largest that PIO can support. Does PIO link to a different standard library, with a different heap manager? If so, that might explain it.

 

[jmarsh]

C:/Users/___/OneDrive/Documents/PlatformIO/Projects/250222-170929-teensy41/src/FastCRC_CRC32.ino:33:100: warning: conversion from 'unsigned int' to 'uint16_t' {aka 'short unsigned int'} changes value from '511872' to '53120' [-Woverflow]

This here is your answer. The length argument of the crc32() function is a uint16_t so it can only process up to 65535 bytes of data. Passing in any value higher will get truncated to 16 bits.

 

[samm_flynn]

This here is your answer. The length argument of the crc32() function is a uint16_t so it can only process up to 65535 bytes of data. Passing in any value higher will get truncated to 16 bits.

I don't use PIO, so I'm guessing, but your DMAMEM array leaves 0 bytes free in RAM2, and perhaps there is something different about the linker script in PIO that results in some of those bytes being used. Have you tried testing with a smaller array? If that works, then you may simply be making the array larger than the largest that PIO can support. Does PIO link to a different standard library, with a different heap manager? If so, that might explain it.

@jmarsh and @joepasquariello , the root of all my problem is in the pio platform size_t is a uint16_t type var, any idea how can i change that. In arduino ide it gets interpreted as a uinjt32_t

I tried with a smaller array, array size of 2730 elements, it works with pio then, which is for me 2.730 seconds worth of data.

 

[samm_flynn]

I found a temporary solution, I downloaded the FASTCRC library and put it it the lib folder of my PIO project, seems to be working now.

 

[shawn]

I’d like to see your platformio.ini file. Size_t does not resolve to uint16_t with PIO’s Teensyduino.

 

[samm_flynn]

@shawn

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html
[env:teensy41]
platform = teensy
board = teensy41
framework = arduino

it gets changed to uint16_t somewhere that's for sure.

 

[shawn]

Where are you seeing size_t being equivalent to a uint16_t?

 

[samm_flynn]

If I hover my mouse cursor over the crc32 method from vscode, and arduino ide, I see two different things

 

[shawn]

My first guess is that they’re probably using different libraries.

 

[samm_flynn]

I think you are right, If I paste the FASTCRC library, downloading directly from github, to the lib folder in the platformio project, it works fine and the function parameters seems to match -

 

[samm_flynn]

My first guess is that they’re probably using different libraries.

I looked into the folder where all the platformio libraries are located for teensy boards, and replaced it with the one from github, things are working now, thanks for the insight @shawn. Now this is working for all my projects.