Reading incoming serial string

Status
Not open for further replies.

RomanP

Member
Hello everyone,

This might have already been answered somewhere but after looking I couldn't find a good answer for my problem.
I have an instrument (POPS) sending data once per second. The data string looks like this:

xdata=3801013B1D00012200A0005900420047004C007A0008

xdata= is packet header
All data are ASCII formatted MSB-first hexadecimal numbers
38 is POPS instrument ID
01 is daisy chain index
013B is total particle concentration for all size bins
1D is 10xflow_rate
00 is 100+temperatureexternal thermistor
0122… is the concentrations for each of the 8 size bins. Each concentration is represented by a 4 digit
hex, so there are 32 green characters in total.


The instrument is connected to the Serial port number 2 on my teensy 3.6. Baud rate is 9600.

What I need from my teensy is to listen and detect when the string begins, read the string and then simply extract the 5th to 8th digits, which correspond to the total particle concentration and assign it to a variable as an integer or float.

Below, you can see the data that is being streamed. I used a TTL to - usb converter directly on my laptop for that.

Capture d’écran (238).png


Thank you very much for your help!
 
Hello everyone,

This might have already been answered somewhere but after looking I couldn't find a good answer for my problem.
I have an instrument (POPS) sending data once per second. The data string looks like this:

xdata=3801013B1D00012200A0005900420047004C007A0008

xdata= is packet header
All data are ASCII formatted MSB-first hexadecimal numbers
38 is POPS instrument ID
01 is daisy chain index
013B is total particle concentration for all size bins
1D is 10xflow_rate
00 is 100+temperatureexternal thermistor
0122… is the concentrations for each of the 8 size bins. Each concentration is represented by a 4 digit
hex, so there are 32 green characters in total.


The instrument is connected to the Serial port number 2 on my teensy 3.6. Baud rate is 9600.

What I need from my teensy is to listen and detect when the string begins, read the string and then simply extract the 5th to 8th digits, which correspond to the total particle concentration and assign it to a variable as an integer or float.

Below, you can see the data that is being streamed. I used a TTL to - usb converter directly on my laptop for that.

View attachment 25687


Thank you very much for your help!

I would write a state machine that parses the packet. You can find an example in my uBlox UBX parser library here:
https://github.com/bolderflight/ublox-arduino/blob/main/src/ublox.cpp#L225

It's reading a byte at a time. If the byte matches what's expected for the packet header, the parser state is advanced, otherwise it's reset to 0. Once the header is identified, the rest of the packet is read into a buffer and then manipulated.
 
When the string is received are there any terminating characters, ie carriage return/Line feed (0D/0A)?
 
I'd just read the string up to the CRLF on the end. First, check that the string is the correct length, otherwise ignore. Then make sure the string starts with 38, otherwise ignore. Read the particle concentration and then wait for the next string.

Pete
 
Dear brtaylor, BriComp, Frank B and el_supremo,

Thank you very much for your answers! I used readStringUntil() as suggest and it works great.

Here's the code I used:

#define POPS_Serial Serial2

void setup() {
Serial.begin(9600);
POPS_Serial.begin(9600);
}

void loop() {


if (POPS_Serial.available() > 0) {
String POPS_string = POPS_Serial.readStringUntil('\n').substring(10,14);
Serial.print("Particle concentration = ");
Serial.println(POPS_string);

}
}

Now, when I read the incoming message, the hex bytes are converted directly into characters. As it turns out, the 4 characters corresponding to the variable "particle concentration" should be regarded as 2 hex bytes which have to be converted into DEC. There I struggle a bit to do this conversion. How would you tell the program that the string 012F is actually 2 hex bytes 01 and 2F and convert that into the corresponding integer, which is 303?

Thank you again for your help, it's very much appreciated!


Capture d’écran (241).jpg
 
RomanP:
You will have to Parse the input string like this:

Code:
/*******************************************************************************************************************
*  This function gets the entire Weather Struct from the slave weather station.
* Needs:
*   The buffer pointer to put the incoming data in (WeatherDataStruct).
*       void* ptr
*
* Returns:
*   A code which tells whether of not the data packet was recieved correctly.
*       Return Code:
*           GOOD_PACKET         = 0
*           BAD_PACKET          = 1
*           PACKET_NOT_FOUND    = 2
*           BAD_CRC             = 3
*           BAD_ACK             = 4
*           BAD_COMMAND         = 5
*           BAD_PREFIX_CHAR     = 6
*           BAD_SUFFIX_CHAR     = 7
*           BAD_DATA_SIZE       = 8
 *******************************************************************************************************************/
packetStatus getDataFromSlave( void *inputDataPointer)
{
    uint8_t CRC1, CRC2;
    uint16_t inputCRC;
    uint8_t i, j;
    
    digitalWrite(13, HIGH);  // This turns on the LED.
    
    #if defined (DEBUG17)
        Serial.print(F("\nThis is the getDataFromSlave Sub. This is the size of the data Returned From Master: "));
        Serial.println(numberOfBytesFromDMAReceive);
        Serial.println(F("Printing the data in the receiveBuffer Buffer..."));
        printBuffer(receiveBuffer, RxBUFSIZE);
    #endif
    
    
    i = 0;  // Start with the Pointer to the input data array at zero.
    j = 0; // Start with the pointer to the receiveBuffer array at zero.
    
    #if defined (DEBUG17)
        Serial.println(F("Begin transferring bytes from transmit buffer to local buffer..."));
        Serial.print(F("This is the value of j: "));
        Serial.println(j, DEC);
        Serial.println(F("Read the prefix character from the transmit Buffer OK..."));
        Serial.print(F("This is the value of the prefix character: "));
        Serial.println( ( receiveBuffer[j] ), DEC);
    #endif
    
    // Receive the data from the slave.
    // The recieved packet is in the receiveBuffer, (Stx), (Command), (Size of Data Packet), (Data...), (CRC1),(CRC2), (Etx).
    // It needs to be parsed for a command byte and good data.
    if( ( receiveBuffer[j] ) != (byte)prefixChar )  // Check for the correct Prefix Character.
        {
            // Prefix character is not found
            #if defined (DEBUG17)
                Serial.println(F("Prefix character is not found!\n"));
            #endif
            digitalWrite(13, LOW);  // This turns off the LED.
            return BAD_PACKET;
        }
    else
        {
            j++; // increment the pointer to the receiveBuffer array.
        
            #if defined (DEBUG17)
                Serial.println(F("Reading the receive Command from the transmit Buffer..."));
                Serial.print(F("This is the value of the Command Code: "));
                Serial.println( ( receiveBuffer[j] ), DEC);
                Serial.print(F("This is the value of j: "));
                Serial.println(j, DEC);
            #endif
        
            receiveCommand = (commandTypes)receiveBuffer[j];  // This is the number of the command recieved from the master to execute.
            if ( !(receiveCommand >= 0 ) && ( receiveCommand < maxCommandNumber ) )
                {
                    #if defined (DEBUG17)
                        Serial.println(F("Receive Command is bad!\n"));
                    #endif
                    digitalWrite(13, LOW);  // This turns off the LED.
                    return BAD_COMMAND;
                }
        
            j++; // increment the pointer to the receiveBuffer array.
        
            #if defined (DEBUG17)
                Serial.println(F("Read the receive Command from the transmit Buffer OK..."));
                Serial.print(F("This is the value of j: "));
                Serial.println(j, DEC);
                Serial.print(F("This is the size of the Command Data Packet: "));
                Serial.println( ( receiveBuffer[j] ), DEC);
            #endif
        
            receivedDataSize = receiveBuffer[j];  // This is the size of the command data packet (<= 127 bytes).
            if ( receivedDataSize > maxRxDataBufferSize )
                {
                    #if defined (DEBUG17)
                        Serial.println(F("Receive Data Size is more than the buffer size!\n"));
                    #endif
                    digitalWrite(13, LOW);  // This turns off the LED.
                    return BAD_DATA_SIZE;
                }
        
            j++; // increment the pointer to the receiveBuffer array.
        
            uint8_t data[receivedDataSize];  // Create a tmp char array of the data from the Uart.
            #if defined (DEBUG17)
                Serial.println(F("Read the receive Data Size from the transmit Buffer OK..."));
                Serial.print(F("This is the value of receivedDataSize: "));
                Serial.println(receivedDataSize, DEC);
                Serial.print(F("This is the value of j: "));
                Serial.println(j, DEC);
            #endif
        
            for( i = 0; i < ( receivedDataSize ); i++)
                {
                    data[i] = (uint8_t)receiveBuffer[j];  // Copy the data from the receive Buffer to the temp byte array.
                    j++; // increment the pointer to the receiveBuffer array.
                }
        
            // Need to get the 2 byte CRC from the next two bytes in the receiveBuffer (DMA data from Uart2).
            /// Need to do the CRC check on these two characters.
            #if defined (DEBUG17)
                Serial.println(F("Transfered the data from the transmit buffer to the temp array..."));
                Serial.print(F("This is the data in the buffer: "));
                printBuffer(data, receivedDataSize);
                Serial.print(F("\nThis is the value of j: "));
                Serial.println(j, DEC);
            #endif
        
            /// Need to move the pointer to the first CRC byte all the way past the 121 byte data packet.
        
            j = j + ( maxRxDataBufferSize - receivedDataSize );
        
            #if defined (DEBUG17)
                Serial.println(F("Moved the j pointer to the end of the data packet..."));
                Serial.print(F("This is the value of j: "));
                Serial.println(j, DEC);
            #endif
        
        
            CRC1 = (uint8_t)receiveBuffer[j];
        
            j++; // increment the pointer to the receiveBuffer array.
        
            #if defined (DEBUG17)
                Serial.print(F("This is the CRC1 byte from the receiveBuffer: "));
                Serial.println(CRC1, HEX);
                Serial.print(F("This is the value of j: "));
                Serial.println(j, DEC);
            #endif
        
            CRC2 = (uint8_t)receiveBuffer[j];
        
            #if defined (DEBUG22)
                Serial.print(F("This is the CRC2 byte from the receiveBuffer: "));
                Serial.println(CRC2, HEX);
            #endif
        
            /// Convert two 8 bit bytes to a 16 bit word.
            inputCRC = 0;  // Start out with zero
            inputCRC = ( ( uint16_t)CRC2 << 8);  // This is the high byte, shift left 8 places and put into the 16 bit inputCRC variable.
            inputCRC = (inputCRC | ( uint16_t)CRC1);
        
            #if defined (DEBUG17)
                Serial.print(F("This is the returned CRC word: "));
                Serial.println(inputCRC, HEX);
            #endif
        
            // bool crcCheck(void * inputBuffer, uint16_t bufferSize , uint16_t inputCRC)
            if ( crcCheck(data, receivedDataSize, inputCRC) == false )
                {
                    // The CRC recieved was bad.
                    #if defined (DEBUG17)
                        Serial.println(F("The CRC recieved was bad!\n"));
                    #endif
                    digitalWrite(13, LOW);  // This turns off the LED.
                    return BAD_CRC;
                }
            else
                {
                    j++; // increment the pointer to the receiveBuffer array.
            
                    #if defined (DEBUG17)
                        Serial.print(F("This is the value of j: "));
                        Serial.println(j, DEC);
                    #endif
            
                    if( receiveBuffer[j] !=  suffixChar)
                        {
                            // Suffix character is not found
                            #if defined (DEBUG17)
                                Serial.println(F("Suffix character is Bad!\n"));
                            #endif
                            digitalWrite(13, LOW);  // This turns off the LED.
                            return BAD_SUFFIX_CHAR;
                        }
                    else
                        {
                            #if defined (DEBUG17)
                                Serial.println(F("Coping the bytes into the struct passed in!"));
                            #endif
                
                            memcpy(inputDataPointer, data, receivedDataSize);  // Copy the bytes from the receiveBuffer (DMA data from Uart2) into the dataReturnedFromMaster Struct.
                
                            #if defined (DEBUG17)
                                Serial.println(F("Good Packet found!\n"));
                            #endif
                            digitalWrite(13, LOW);  // This turns off the LED.
                            return GOOD_PACKET;
                        }
                }
        }
}

I used this technique to parse a stream of data from an outside weather station. This code has been working continuously for about 7 years.
As you might surmise this stream sort-of mimics an ethernet packet complete with start character, stop character, command type, data packet size, data packet, and CRC. I'm sure you could easily shorten this for your needs.

Regards,
Ed
 
Status
Not open for further replies.
Back
Top