Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 7 of 7

Thread: Reading incoming serial string

  1. #1
    Junior Member
    Join Date
    Jun 2020
    Posts
    12

    Reading incoming serial string

    Hello everyone,

    This might have already been answered somewhere but after looking I couldn't find a good answer for my problem.
    I have an instrument (POPS) sending data once per second. The data string looks like this:

    xdata=3801013B1D00012200A0005900420047004C007A0008

    xdata= is packet header
    All data are ASCII formatted MSB-first hexadecimal numbers
    38 is POPS instrument ID
    01 is daisy chain index
    013B is total particle concentration for all size bins
    1D is 10xflow_rate
    00 is 100+temperatureexternal thermistor
    0122… is the concentrations for each of the 8 size bins. Each concentration is represented by a 4 digit
    hex, so there are 32 green characters in total.


    The instrument is connected to the Serial port number 2 on my teensy 3.6. Baud rate is 9600.

    What I need from my teensy is to listen and detect when the string begins, read the string and then simply extract the 5th to 8th digits, which correspond to the total particle concentration and assign it to a variable as an integer or float.

    Below, you can see the data that is being streamed. I used a TTL to - usb converter directly on my laptop for that.

    Click image for larger version. 

Name:	Capture d’écran (238).png 
Views:	30 
Size:	616.6 KB 
ID:	25687


    Thank you very much for your help!

  2. #2
    Senior Member brtaylor's Avatar
    Join Date
    Mar 2016
    Location
    Portland, OR
    Posts
    711
    Quote Originally Posted by RomanP View Post
    Hello everyone,

    This might have already been answered somewhere but after looking I couldn't find a good answer for my problem.
    I have an instrument (POPS) sending data once per second. The data string looks like this:

    xdata=3801013B1D00012200A0005900420047004C007A0008

    xdata= is packet header
    All data are ASCII formatted MSB-first hexadecimal numbers
    38 is POPS instrument ID
    01 is daisy chain index
    013B is total particle concentration for all size bins
    1D is 10xflow_rate
    00 is 100+temperatureexternal thermistor
    0122… is the concentrations for each of the 8 size bins. Each concentration is represented by a 4 digit
    hex, so there are 32 green characters in total.


    The instrument is connected to the Serial port number 2 on my teensy 3.6. Baud rate is 9600.

    What I need from my teensy is to listen and detect when the string begins, read the string and then simply extract the 5th to 8th digits, which correspond to the total particle concentration and assign it to a variable as an integer or float.

    Below, you can see the data that is being streamed. I used a TTL to - usb converter directly on my laptop for that.

    Click image for larger version. 

Name:	Capture d’écran (238).png 
Views:	30 
Size:	616.6 KB 
ID:	25687


    Thank you very much for your help!
    I would write a state machine that parses the packet. You can find an example in my uBlox UBX parser library here:
    https://github.com/bolderflight/ublo...ublox.cpp#L225

    It's reading a byte at a time. If the byte matches what's expected for the packet header, the parser state is advanced, otherwise it's reset to 0. Once the header is identified, the rest of the packet is read into a buffer and then manipulated.

  3. #3
    Senior Member BriComp's Avatar
    Join Date
    Apr 2014
    Location
    Cheltenham, UK
    Posts
    416
    When the string is received are there any terminating characters, ie carriage return/Line feed (0D/0A)?

  4. #4
    Senior Member
    Join Date
    Apr 2014
    Location
    Germany
    Posts
    9,333
    Quote Originally Posted by BriComp View Post
    When the string is received are there any terminating characters, ie carriage return/Line feed (0D/0A)?
    According to the picture, yes.

    I think the easiest is readStringUntil()

  5. #5
    Senior Member
    Join Date
    Nov 2012
    Posts
    1,750
    I'd just read the string up to the CRLF on the end. First, check that the string is the correct length, otherwise ignore. Then make sure the string starts with 38, otherwise ignore. Read the particle concentration and then wait for the next string.

    Pete

  6. #6
    Junior Member
    Join Date
    Jun 2020
    Posts
    12
    Dear brtaylor, BriComp, Frank B and el_supremo,

    Thank you very much for your answers! I used readStringUntil() as suggest and it works great.

    Here's the code I used:

    #define POPS_Serial Serial2

    void setup() {
    Serial.begin(9600);
    POPS_Serial.begin(9600);
    }

    void loop() {


    if (POPS_Serial.available() > 0) {
    String POPS_string = POPS_Serial.readStringUntil('\n').substring(10,14) ;
    Serial.print("Particle concentration = ");
    Serial.println(POPS_string);

    }
    }

    Now, when I read the incoming message, the hex bytes are converted directly into characters. As it turns out, the 4 characters corresponding to the variable "particle concentration" should be regarded as 2 hex bytes which have to be converted into DEC. There I struggle a bit to do this conversion. How would you tell the program that the string 012F is actually 2 hex bytes 01 and 2F and convert that into the corresponding integer, which is 303?

    Thank you again for your help, it's very much appreciated!


    Click image for larger version. 

Name:	Capture d’écran (241).jpg 
Views:	9 
Size:	95.0 KB 
ID:	25718

  7. #7
    Member
    Join Date
    Jul 2014
    Location
    Currently Odessa, Ukraine
    Posts
    35
    RomanP:
    You will have to Parse the input string like this:

    Code:
    /*******************************************************************************************************************
    *  This function gets the entire Weather Struct from the slave weather station.
    * Needs:
    *   The buffer pointer to put the incoming data in (WeatherDataStruct).
    *       void* ptr
    *
    * Returns:
    *   A code which tells whether of not the data packet was recieved correctly.
    *       Return Code:
    *           GOOD_PACKET         = 0
    *           BAD_PACKET          = 1
    *           PACKET_NOT_FOUND    = 2
    *           BAD_CRC             = 3
    *           BAD_ACK             = 4
    *           BAD_COMMAND         = 5
    *           BAD_PREFIX_CHAR     = 6
    *           BAD_SUFFIX_CHAR     = 7
    *           BAD_DATA_SIZE       = 8
     *******************************************************************************************************************/
    packetStatus getDataFromSlave( void *inputDataPointer)
    {
        uint8_t CRC1, CRC2;
        uint16_t inputCRC;
        uint8_t i, j;
        
        digitalWrite(13, HIGH);  // This turns on the LED.
        
        #if defined (DEBUG17)
            Serial.print(F("\nThis is the getDataFromSlave Sub. This is the size of the data Returned From Master: "));
            Serial.println(numberOfBytesFromDMAReceive);
            Serial.println(F("Printing the data in the receiveBuffer Buffer..."));
            printBuffer(receiveBuffer, RxBUFSIZE);
        #endif
        
        
        i = 0;  // Start with the Pointer to the input data array at zero.
        j = 0; // Start with the pointer to the receiveBuffer array at zero.
        
        #if defined (DEBUG17)
            Serial.println(F("Begin transferring bytes from transmit buffer to local buffer..."));
            Serial.print(F("This is the value of j: "));
            Serial.println(j, DEC);
            Serial.println(F("Read the prefix character from the transmit Buffer OK..."));
            Serial.print(F("This is the value of the prefix character: "));
            Serial.println( ( receiveBuffer[j] ), DEC);
        #endif
        
        // Receive the data from the slave.
        // The recieved packet is in the receiveBuffer, (Stx), (Command), (Size of Data Packet), (Data...), (CRC1),(CRC2), (Etx).
        // It needs to be parsed for a command byte and good data.
        if( ( receiveBuffer[j] ) != (byte)prefixChar )  // Check for the correct Prefix Character.
            {
                // Prefix character is not found
                #if defined (DEBUG17)
                    Serial.println(F("Prefix character is not found!\n"));
                #endif
                digitalWrite(13, LOW);  // This turns off the LED.
                return BAD_PACKET;
            }
        else
            {
                j++; // increment the pointer to the receiveBuffer array.
            
                #if defined (DEBUG17)
                    Serial.println(F("Reading the receive Command from the transmit Buffer..."));
                    Serial.print(F("This is the value of the Command Code: "));
                    Serial.println( ( receiveBuffer[j] ), DEC);
                    Serial.print(F("This is the value of j: "));
                    Serial.println(j, DEC);
                #endif
            
                receiveCommand = (commandTypes)receiveBuffer[j];  // This is the number of the command recieved from the master to execute.
                if ( !(receiveCommand >= 0 ) && ( receiveCommand < maxCommandNumber ) )
                    {
                        #if defined (DEBUG17)
                            Serial.println(F("Receive Command is bad!\n"));
                        #endif
                        digitalWrite(13, LOW);  // This turns off the LED.
                        return BAD_COMMAND;
                    }
            
                j++; // increment the pointer to the receiveBuffer array.
            
                #if defined (DEBUG17)
                    Serial.println(F("Read the receive Command from the transmit Buffer OK..."));
                    Serial.print(F("This is the value of j: "));
                    Serial.println(j, DEC);
                    Serial.print(F("This is the size of the Command Data Packet: "));
                    Serial.println( ( receiveBuffer[j] ), DEC);
                #endif
            
                receivedDataSize = receiveBuffer[j];  // This is the size of the command data packet (<= 127 bytes).
                if ( receivedDataSize > maxRxDataBufferSize )
                    {
                        #if defined (DEBUG17)
                            Serial.println(F("Receive Data Size is more than the buffer size!\n"));
                        #endif
                        digitalWrite(13, LOW);  // This turns off the LED.
                        return BAD_DATA_SIZE;
                    }
            
                j++; // increment the pointer to the receiveBuffer array.
            
                uint8_t data[receivedDataSize];  // Create a tmp char array of the data from the Uart.
                #if defined (DEBUG17)
                    Serial.println(F("Read the receive Data Size from the transmit Buffer OK..."));
                    Serial.print(F("This is the value of receivedDataSize: "));
                    Serial.println(receivedDataSize, DEC);
                    Serial.print(F("This is the value of j: "));
                    Serial.println(j, DEC);
                #endif
            
                for( i = 0; i < ( receivedDataSize ); i++)
                    {
                        data[i] = (uint8_t)receiveBuffer[j];  // Copy the data from the receive Buffer to the temp byte array.
                        j++; // increment the pointer to the receiveBuffer array.
                    }
            
                // Need to get the 2 byte CRC from the next two bytes in the receiveBuffer (DMA data from Uart2).
                /// Need to do the CRC check on these two characters.
                #if defined (DEBUG17)
                    Serial.println(F("Transfered the data from the transmit buffer to the temp array..."));
                    Serial.print(F("This is the data in the buffer: "));
                    printBuffer(data, receivedDataSize);
                    Serial.print(F("\nThis is the value of j: "));
                    Serial.println(j, DEC);
                #endif
            
                /// Need to move the pointer to the first CRC byte all the way past the 121 byte data packet.
            
                j = j + ( maxRxDataBufferSize - receivedDataSize );
            
                #if defined (DEBUG17)
                    Serial.println(F("Moved the j pointer to the end of the data packet..."));
                    Serial.print(F("This is the value of j: "));
                    Serial.println(j, DEC);
                #endif
            
            
                CRC1 = (uint8_t)receiveBuffer[j];
            
                j++; // increment the pointer to the receiveBuffer array.
            
                #if defined (DEBUG17)
                    Serial.print(F("This is the CRC1 byte from the receiveBuffer: "));
                    Serial.println(CRC1, HEX);
                    Serial.print(F("This is the value of j: "));
                    Serial.println(j, DEC);
                #endif
            
                CRC2 = (uint8_t)receiveBuffer[j];
            
                #if defined (DEBUG22)
                    Serial.print(F("This is the CRC2 byte from the receiveBuffer: "));
                    Serial.println(CRC2, HEX);
                #endif
            
                /// Convert two 8 bit bytes to a 16 bit word.
                inputCRC = 0;  // Start out with zero
                inputCRC = ( ( uint16_t)CRC2 << 8);  // This is the high byte, shift left 8 places and put into the 16 bit inputCRC variable.
                inputCRC = (inputCRC | ( uint16_t)CRC1);
            
                #if defined (DEBUG17)
                    Serial.print(F("This is the returned CRC word: "));
                    Serial.println(inputCRC, HEX);
                #endif
            
                // bool crcCheck(void * inputBuffer, uint16_t bufferSize , uint16_t inputCRC)
                if ( crcCheck(data, receivedDataSize, inputCRC) == false )
                    {
                        // The CRC recieved was bad.
                        #if defined (DEBUG17)
                            Serial.println(F("The CRC recieved was bad!\n"));
                        #endif
                        digitalWrite(13, LOW);  // This turns off the LED.
                        return BAD_CRC;
                    }
                else
                    {
                        j++; // increment the pointer to the receiveBuffer array.
                
                        #if defined (DEBUG17)
                            Serial.print(F("This is the value of j: "));
                            Serial.println(j, DEC);
                        #endif
                
                        if( receiveBuffer[j] !=  suffixChar)
                            {
                                // Suffix character is not found
                                #if defined (DEBUG17)
                                    Serial.println(F("Suffix character is Bad!\n"));
                                #endif
                                digitalWrite(13, LOW);  // This turns off the LED.
                                return BAD_SUFFIX_CHAR;
                            }
                        else
                            {
                                #if defined (DEBUG17)
                                    Serial.println(F("Coping the bytes into the struct passed in!"));
                                #endif
                    
                                memcpy(inputDataPointer, data, receivedDataSize);  // Copy the bytes from the receiveBuffer (DMA data from Uart2) into the dataReturnedFromMaster Struct.
                    
                                #if defined (DEBUG17)
                                    Serial.println(F("Good Packet found!\n"));
                                #endif
                                digitalWrite(13, LOW);  // This turns off the LED.
                                return GOOD_PACKET;
                            }
                    }
            }
    }
    I used this technique to parse a stream of data from an outside weather station. This code has been working continuously for about 7 years.
    As you might surmise this stream sort-of mimics an ethernet packet complete with start character, stop character, command type, data packet size, data packet, and CRC. I'm sure you could easily shorten this for your needs.

    Regards,
    Ed

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •