converting double from 8 bytes from a gps sensor

Status
Not open for further replies.

jasper

Active member
i cant really find a definitive answer as to how to go from 8 bytes into a double, i have tried two methods based on my limited knowlege and experience of what what works with longs and floats, but these methods don't seem to work for double.

my goal is to get data that looks like a GPS reading something like "30.111111" and "-29.000000" (my office in South Africa), and the actual data received is also shown in the example below. if i can just get the double conversion to work on a plain simple 8 bytes then i can fiddle around the GPS bytes with more confidence.

here is the sketch and two methods, both work for long but none seem to work for double!!!

Code:
	// pass in 4 bytes returns a long
	long BytesToLong(byte b0, byte b1, byte b2, byte b3){
		    long iresult = b0; 
			iresult = iresult + 256 * b1; 
			iresult = iresult + 256 *256 * b2;  			 
			iresult = iresult + 256 * 256 * 256 * b3;  
			return iresult;
		}
	
	//pass in 4 bytes returns a long uses union method
	long BytesTolu(byte b0, byte b1, byte b2, byte b3){
			union u_tag { byte b[4]; long fval;} u; // temporary variable to convert bytes to long
			u.b[0] =b0;		
			u.b[1] = b1; 		
			u.b[2] = b2; 		
			u.b[3] = b3; 
			return u.fval;
		}

	//pass in 8 bytes returns a double uses union method
	double BytesTodu(byte b0, byte b1, byte b2, byte b3, byte b4, byte b5, byte b6, byte b7){
			union u_tag { byte b[8]; double fval;} u; // temporary variable to convert bytes to double
			u.b[0] =b0;		
			u.b[1] = b1; 		
			u.b[2] = b2; 		
			u.b[3] = b3; 		
			u.b[4] = b4;		
			u.b[5] = b5; 		
			u.b[6] = b6; 		
			u.b[7] = b7; 
			return u.fval;
		}

//pass in 8 bytes returns a double
double BytesToDouble(byte b0, byte b1, byte b2, byte b3, byte b4, byte b5, byte b6, byte b7){
    double iresult = b0; 
			iresult = iresult + 256 * b1; 
			iresult = iresult + 256 *256 * b2;  			 
			iresult = iresult + 256 * 256 * 256 * b3;  	
			iresult = iresult + 256 * 256 * 256 * 256 * b4;  		 
			iresult = iresult + 256 * 256 * 256 * 256 * 256 * b5;  		 
			iresult = iresult + 256 * 256 * 256 * 256 * 256 * 256 * b6;  
			iresult = iresult + 256 * 256 * 256 * 256 * 256 * 256 * 256 * b7;  
			return iresult;
		}

void setup()
{
Serial.begin (115200);
delay(500);// short delay to allow com port to initialise
Serial.println ("starting...");

// testing long
 byte b[4] = {0,0,0,1};  // our byte array recevived from some sensor
 long iresult = BytesToLong(b[0],b[1],b[2],b[3]);
 Serial.print("Long using multip:"); Serial.println(iresult);

 iresult = BytesTolu(b[0],b[1],b[2],b[3]);
 Serial.print("Long using union:"); Serial.println(iresult);

 // testing double 
 byte d[8] = {0,0,0,1,0,0,0,0};  // an 8 byte array recevived from some other sensor
 double dresult = BytesToDouble(d[0],d[1],d[2],d[3],d[4],d[5],d[6],d[7]);
 Serial.print("double using multip:"); Serial.println(dresult);

 dresult = BytesTodu(d[0],d[1],d[2],d[3],d[4],d[5],d[6],d[7]);
 Serial.print("double using union:"); Serial.println(dresult);

 // actual  readings from an actual sensor
 byte longitude[8] = {107	,80	,12	,26,	2,	163,	224,	191};
 byte latitude[8] = {74,	218	,254	,192	,140	,45,	225	,63};

 double longi, lati;
 longi = BytesToDouble(longitude[0], longitude[1],longitude[2],longitude[3],longitude[4],longitude[5],longitude[6],longitude[7]);
 Serial.print("longitude:"); Serial.println(longi);
  // longi should be 31.0

 lati = BytesToDouble(latitude[0], latitude[1],latitude[2],latitude[3],latitude[4],latitude[5],latitude[6],latitude[7]);
 Serial.print("latitude:"); Serial.println(lati);
   // latti should be -29.0


}
 
This is impossible to answer until we know the exact format/meaning of thoose 8 bytes.
Are they really a double transported one byte at a time, or the location packed in some other way.

A reference to the sensor datasheet could help.
 
Agreed, without reliable tech specs on the data format, this is a guessing game.

We also don't know which Teensy you're using. That matters, because the older 8 bit AVR chips don't natively support 64 bit double. The 32 bit ARM boards do, but for printing numbers, you need certain settings and code to make proper use of 64 bit double, without conversion to 32 bit float. I could write a much longer answer... but maybe you could first post the missing info, to save everyone's time?
 
Is the GPS data is the common NEMA0183 formatted ASCII text? If so there are lots of examples on the 'net.
A common method is to use sscanf() with a 32 bit float.. that has enough precision for GPS lat/lon accuracy. You'll need the compiler and run time library options set to permit floats in printf and sscanf et al.
 
Is the GPS data is the common NEMA0183 formatted ASCII text? If so there are lots of examples on the 'net.
A common method is to use sscanf() with a 32 bit float.. that has enough precision for GPS lat/lon accuracy. You'll need the compiler and run time library options set to permit floats in printf and sscanf et al.

The examples in the code are not NMEA values (there is a 2 in one of the bytes).

There is a binary format defined for GPS, but as long jasper does not provide info on actual device, mode setting or reference to protocol, and type of host computer he got the lat/long values from, it is not easy to reverse engineer, the formula.

Also -29.0 and 31.0 in double (real8 according to GPS specs) should not be the values given in the code
according to my tests (num2hex in matlab)

31.0 becomes in hex 40 3f 00 00 00 00 00 00
-29.0 becomes in hex c0 3d 00 00 00 00 00 00
 
ThanksT for all the input. The documentation is in chinglish, it just says latitude : "double"
This is just 2 sections of a long data string, rest assured all of the other float, long and integer sections worked well using those techniques I demonstrate in the code, there's a bunch of stuff coming from the drone that we are interpreting correctly using those techniques, height, altitude, satellite count, radio control inputs, etc. We choked on the double , at the moment we are just skipping past those 16 bytes.

Dont get hung up on the gps stuff, maybe I shouldn't have included that. The point of this post is to establish if my techniques are valid for assembling a double from 8 bytes. It seems to be a fail even on the simplest example shown of 8 bytes: 0 0 0 0 0 0 0 1

I forgot to mention lsb on the left, array position 0 so { 1, 0, 0, 0, 0, 0, 0, 0} represents 1

Its for teensy 3.1 nothing less.
 
The point of this post is to establish if my techniques are valid for assembling a double from 8 bytes.

No. Your techniques are not valid.

Look at your code. Are there any decimals in them? Would you expect it to give a valid result. Do the math with pen and paper if uncertain.


Binary coded floats are in what could easiest be described as scientific notation. That is mantissa * 2^exponent, and not as 3.1*10^1 as we humans tend to read learn from young age. Look up references for IEEE-754 which is the most common floating point format.

Even if your code had just mapped the bytes directly onto the memory representation of a float (which you do not do) the result would not be the numbers you suggest. I sat down for a while and tried to map your bytes to something reasonable but failed.

Are you sure the byte arrays you have in the sample code are correct? Is it really exactly the numbers you have in the code (or in your text)?

Some GPS receivers can do tricks to output coordinates in other coordinate systems. Perhaps that is happening.

Do you have a GPS lock? I have seen devices outputting random data when started and keeping these until a valid fix has been calculated.
 
Lets assume its some chinese co-ordinate system which we cant fathom, my question still is, how will I convert the bytes into double , assuming the byte order is correct , which i think it is because all of their other information is consistent , including the subsequent bytes in the package being height, altitude and GPS health (we are getting 4 satelites indoors) for which i use those techniques to make up floats and integer and they work pefectly - we fly the drone up to about 5m and the readout is 4.8m and the drone's app displays 4.8m as well.

To be honest we dont really need the GPS , its the only "doubles" in their entire frame of 200 bytes, it just perplexed me that my logic works for floats and integers but didnt work for doubles.

Here's a video, its using a arduino mega in the video for testing, (without the gps) but our secret weapon is the teensy. dont want to give away too many trade secrets!!!
https://www.youtube.com/watch?v=_9VR9IQP7QM
 
In principle the "union" method is sound, telling the compiler that a sequence of 8 bytes is in reality the eight bytes of a double.

Of course this assumes correct byte ordering, and that the data is really a double in the same format as the compiler/mcu expects. As have been pointed out your byte sequence given in the code does not represent the any of the numbers 31.someting or -29.something in standard double floating ponit format.

I guess this is as much help as you can get.
 
If I were you, I'd just connect the GPS to the Teensy, load a TinyGPS example and set correct rx/tx.
Further you could also try TinyGPS++, which is an even more professional library. Both of them have done great with any module I have used and they're well documented and known to work on Teensy.

...the new library can extract arbitrary data from any of the myriad NMEA sentences out there, even proprietary ones.

Unless there's something between the Teensy and GPS, like a proprietary CANbus or something, but then the provided hardware information is insufficient.

Edit: Oh ok, now I watched the video.
Does the Matrice have an A2 or Naza? There has been quite a bit of work done to decode the CAN messages from Naza and get the coordinates. Check RCgroups for "pawelsky". He's on pjrc too now and then.
 
Last edited:
Thanks for the tips. It has a new processor called the n1. Not sure if it is based on the naza . There are plenty of can busses on there, I will try and get some pawelsky Code and see if it makes sense. Our requirements are to actually control the drone, hence we went in through the serial port since that is the documented route, I will check if can bus control is possible.
 
I agree with mlu, a union is the proper way to convert an array of bytes to double, if the bytes are indeed binary data of a IEEE standard double precision floating point number.
 
I agree with mlu, a union is the proper way to convert an array of bytes to double, if the bytes are indeed binary data of a IEEE standard double precision floating point number.

And putting on my compiler writer hat for a moment. Do not play pointer games where you take the address of the double and convert it to be an unsigned byte address. This is not covered by the appropriate ISO language standards, and it can be a source of mystifying bugs when the compiler moves things around. While we have stomped out many of these alias bugs in GCC, I'm sure there are more subtle ones that can pop up.

To quote a signature from Henry Spencer from many years ago: Do not lie to the compiler, for it will get its revenge. :cool:

If you need it, here is the IEEE 754R format for 64-bit double precision: https://en.wikipedia.org/wiki/Double-precision_floating-point_format.

If you are converting code from an Arduino on an ATmel AVR platform, note that the AVR gcc compiler maps the double keyword into float.
 
Last edited:
Please can we step back a bit and look at the simplest example of :
byte d[8] = {0,0,0,0,0,0,0,1};

(where array position 7 is the Most significant Byte).

What technique would you use to convert that to double, and what would the expected result be?
with my logic (that I studied in electronic engineering in 1988 when we only used assembler and fortran), I would say:

256*256*256*256*256*256*256*1 = 72057594037927900 (as calculated by Excel)

can we agree on this?
If so, what code would produce this outcome on teensy 3.1 because my code above does not.
 
You must decide/differentiate if this is a binary (integer) number that shall be converted to double OR
if this is the binary representation of a double according to the IEEE standard.

In the first case, and this corresponds to the case that your sensor returns not a double but an integer.

Your number, in hexadecimal 0x100000000000000 ( a truly big one ) can be converted to double as
1*256.0*256.0* .... *256.0
Observe, this cannot be done with integer arithmethic because it will overflow. So it must be done with float/double datatypes.

In the second case, if your number if it is a proper representation of a double, then it can be converted with the "union"
method. (( update fixed a few extra 0's ))
 
Last edited:
Please can we step back a bit and look at the simplest example of :
byte d[8] = {0,0,0,0,0,0,0,1};

(where array position 7 is the Most significant Byte).

What technique would you use to convert that to double, and what would the expected result be?
with my logic (that I studied in electronic engineering in 1988 when we only used assembler and fortran), I would say:

256*256*256*256*256*256*256*1 = 72057594037927900 (as calculated by Excel)

can we agree on this?
If so, what code would produce this outcome on teensy 3.1 because my code above does not.
No, because that is not how doubles are encoded. In IEEE 754R, a 64-bit double has three fields:

  • A single bit that gives the sign of the value;
  • An 11-bit field that gives a biased exponent (the bias adds 1023 to the exponent, so that 0.0 has a zero sign, zero exponent, and zero mantissa);
  • A 52-bit field that gives the mantissa. For normal non-zero numbers, the top bit is assumed to be 1 and is omitted, so you get 53-bit precision.

In your example, an array of 8 bytes with the most significant byte 1 and the remaining bytes being zero, it is 7.29112e-304 when those bytes are used in a union with a double on a little endian system.

If instead the bytes are given in a big endian format, the byte value would be 4.94066e-324, which is what is called a denormal number (denormals are too small to be represented in full 53 bit precision, and the exponent bits are 0, and the mantissa holds the bits that can be represented.
 
Last edited:
Thanks I'll do some more testing. I see they updated their documentation and said the gps reading is in radians so that's some help!
 
Status
Not open for further replies.
Back
Top