Problem operand with float

[Sandroelec]

Hi all,

i've written a simple code

float K_longitude ;
float longitude;
longitude=45.0;
K_longitude =(longitude+(1.2/10000.0));
Serial.println(K_longitude);

the result was 45.0001182

After i tried to do the same operation with double variables:

doubleK_longitude ;
double longitude;
longitude=45.0;
K_longitude =(longitude+(1.2/10000.0));
Serial.println(K_longitude,7);

at the end the result was correct 45.0001200
I need the result in a float variable because i've to send this value through a serial port.
How is possble to do that?
Why the same operation using floating point variables doesn't work? the number of decimals that i need it's 6 and i think it's good for float variables it's right?

Many thanks

Sandro

 

[AlanK]

Hi,
Round-off errors in floats limit the accuracy to roughly seven significant figures - not seven decimal places

 

[Sandroelec]

Hi,
Round-off errors in floats limit the accuracy to roughly seven significant figures - not seven decimal places

Thanks for your answer, now i do the calculation with double variables and at the end i take the result with a float, how it's possible to truncate double decimals to obtain seven significant figures and convert that?
I tried to do conversion using this metod float variable = (float) doublevariable; but it doesn't work, the result s not good.

Thanks

 

[AlanK]

Hi,
I would expect you to get the same result as before -

45.0001182 or

45.0001183. You won't do any better than that if you convert the result to a float.
If you need more accuracy that that, then you'll need send the values as doubles (or as strings if speed is not critical).
​

​

 

[Sandroelec]

ok so it's no possible truncate the double value to obtain a precise float with less number? Doing all the calculation with double variable which is the maximum value of signifant figures? I'm using teensy 3.6.

 

[mborgerson]

As you have discovered, a single-precision float is only good to about 7 significant figures. That's due to the limitations of having only 23 (or 24, depending on normalization) bits for the mantissa. Thus the mantissa is limited to a precision of 1 part in 16,777,216.

I'm struggling to find an application where I would need to know longitude or latitude in degrees to more than 7 significant digits. Unless you have a survey-quality GPS that you can allow to collect data for several minutes, you will never be able to measure longitude or latitude in degrees to better than about 5 decimal places (equivalent to about 1 meter at the equator).
That five decimal places results in values with seven significant digits and you should be OK for longitudes up to 90 degrees.

There's a discussion of numeric precision and geographic coordinates at:

https://gis.stackexchange.com/questions/8650/measuring-accuracy-of-latitude-and-longitude