Progmem

brtaylor

brtaylor

Well-known member
Hi all -

I'm seeing if I can implement an EGM96 geoid model on Teensy 4.x. Specifically, I'm implementing this library:
https://github.com/emericg/EGM96

I renamed the EGM96.c file to a .cpp file and moved the EGM96_data.h to a EGM96_data.cpp file and changed the header file to be:

Code: 
extern const double egm96_data[65342][4];

The data set itself, in the cpp file is declared as:

Code: 
const double egm96_data[65342][4] PROGMEM = {...}

I'm testing it with the following code:

Code: 
#include "EGM96.h"

void setup() {
  Serial.begin(115200);
  while (!Serial) {}
  double alt = egm96_compute_altitude_offset(35.691544, -105.944183);
  Serial.println(alt);
}

void loop() {}

Compiles fine with the following stats:

Code: 
Memory Usage on Teensy MicroMod:
  FLASH: code:16420, data:2094936, headers:8316   free for files:14395400
   RAM1: variables:28224, code:14720, padding:18048   free for local variables:463296
   RAM2: variables:12384  free for malloc/new:511904

The serial monitor continuously flashes on and off - indicating a power cycle of the board. I've tried this on USB power and with an external power supply. Also tried it with Teensy MMOD and Teensy 4.1. Same symptoms on each. Wondering if I'm doing something wrong with PROGMEM since it's my first time using it.

EDIT: attached a zip of the Arduino program.

View attachment sketch_dec14a.zip
 
Last edited:
Are the restarts after some 8 second delay?

If so adding CrashReport() in setupwill give some idea of the nature of the fault and where.

PROGMEM says 'store on flash. This would be a read only memory area:
PROGMEM & F() - Variables defined with PROGMEM, and strings surrounded by F() are placed only in the flash memory. They can be accessed normally, special functions normally used on 8 bit boards are not required to read PROGMEM variables.

That memory could be marked as DMAMEM in RAM2 and be writeable and not be in primary RAM1 memory.
 
Saw the {...} as not initiated - now seeing it was marked CONST so it must be initialized and not expected to change at runtime?

Though adding CrashReport()'ing and a quick change to DMAMEM might give some info
 
defragster said:
Saw the {...} as not initiated - now seeing it was marked CONST so it must be initialized and not expected to change at runtime?

Though adding CrashReport()'ing and a quick change to DMAMEM might give some info
Click to expand...

Correct, initialized and not expected to change; just a massive array. DMAMEM overflowed. I'm getting a "no match for call to '(CrashReportClass)". I'll have to look more into how using CrashReport works.
 
@brtaylor - wonder if there is a boundary getting crossed - or maybe it needs 8 byte aligned?

some variation of: { double arr[4] __attribute__((aligned(32))); }
 
Morning all

Been playing with this for the couple of hours trying to do some debugging.

First, crashreport is failing looks like as a result of Serial rebooting or recycling too fast.

Did manage to trace it I think to the function hundu. Too much going on in there.

Anyway just as a sanity check I printed the first and last 5 of the egm96_dat array:
Code: 
Testing
0.000000000000000, 0.000000000000000, 0.000000000000000, 0.000000000000000, 
-5.027449999999999, 0.000000000000000, 0.000000000000000, 0.000000000000000, 
0.362816000000000, 0.000000000000000, 0.000000000000000, 0.000000000000000, 
-1.043490000000000, -2.046260000000000, 0.000000000000000, 0.000000000000000, 
-3.184950000000000, 0.000000000000000, 0.000000001403250, 0.000000000000000, 

=====================

-0.000218956000000, 0.002460729999999, -0.000000000020177, 0.000000000005495, 
-0.005546379999999, -0.001383930000000, 0.000000000043706, -0.000000000104331, 
-0.001192900000000, -0.001789100000000, -0.000000000006280, 0.000000000106636, 
-0.004444240000000, -0.006218829999999, 0.000000000070960, 0.000000000069176, 
-0.007386490000000, 0.000547503999999, 0.000000000018397, -0.000000000031012, 
0.000000000000000, 0.007252469999999, -0.000000000000000, -0.000000000083022,
which pretty much matches what you setup.

So probably have to check whats getting into hundu and then whats happening with the calculations.
 
Ok since I can't resist. I just changed function undulation to:
Code: 
double undulation(double lat, double lon)
{
    double p[_coeffs+1], sinml[_361+1], cosml[_361+1], rleg[_361+1];

    double rlat, gr, re;
    unsigned nmax1 = _nmax + 1;

    // compute the geocentric latitude, geocentric radius, normal gravity
    radgra(lat, lon, &rlat, &gr, &re);
    rlat = (M_PI / 2) - rlat;

    Serial.println(_coeffs);
    Serial.println(rlat); Serial.println(gr); Serial.println(re);
    
    for (unsigned j = 1; j <= nmax1; j++)
    {
        unsigned m = j - 1;
        legfdn(m, rlat, rleg);
        for (unsigned i = j ; i <= nmax1; i++)
        {
            p[(((i - 1) * i) / 2) + m + 1] = rleg[i];
            Serial.print((((i - 1) * i) / 2) + m + 1); Serial.print(",  "); 
            Serial.print(rleg[i]); Serial.print(",  ");
            Serial.println(p[(((i - 1) * i) / 2) + m + 1]);
        }
     }
     //for(unsigned i = 2; i < 10; i++){
    //  Serial.println(p[i]);
    // }

     dscml(lon, sinml, cosml);
    return -0.53;
     //return hundu(p, sinml, cosml, gr, re);
}
and it will give me:
Code: 
Testing
65341
0.95
9.80
6370898.33
1,  1.00,  1.00
2,  1.01,  1.01
4,  0.01,  0.01
7,  -1.01,  -1.01
11,  -1.18,  -1.18
16,  -0.35,  -0.35
22,  0.78,  0.78
29,  1.25,  1.25
37,  0.67,  0.67
46,  -0.47,  -0.47
....
64979,  0.00,  0.00
65339,  0.00,  0.00
64980,  0.00,  0.00
65340,  0.00,  0.00
65341,  0.00,  0.00

looks good so far right, but if you uncomment the print loop for "p" it goes into that infinite loop again.
 
Ok last couple of comments.

1. Just realized that his indexes go from 1 to xxxxx. Ok fortran translation. He handles it by adjusting the array dimensions.

2. Chaning all the doubles to floats works:
Code: 
Output:
-9.65

3. Tried using a T4.1 with that large array in extmem and still would crash serial.
 
I tried aligning the array with the following and no improvement. Mike, I need to think about it some, but your solution using floats is probably the best - I think EGM96 doesn't need doubles.

Code: 
__attribute__((aligned(32)))

Shoot, this was a quick thing at the end of the day yesterday to see if I could have an easy way of getting a good estimate of MSL altitude and it's turned into a can of worms :)
 
brtaylor said:
I tried aligning the array with the following and no improvement. Mike, I need to think about it some, but your solution using floats is probably the best - I think EGM96 doesn't need doubles.

Code: 
__attribute__((aligned(32)))

Shoot, this was a quick thing at the end of the day yesterday to see if I could have an easy way of getting a good estimate of MSL altitude and it's turned into a can of worms :)
Click to expand...

No good deed :) But i am wondering why its failing - not smart enough to figure it out :)
 
@brtaylor

Got the doubles version working using a T4.1 with EXTMEM.

What I did was make the p[_coeffs+1] global and assigned it to extmem and I did get an output from it:
Code: 
Output:
-18.92
which is larger than using the float version by almost 2. Don't know which one is right though.

Think the issue was probably a runtime space allocation error since that p array is 65K but only gets allocated when the the function is called - just a wild guess.

And yes I am a glutton for punishment
 
Thanks Mike! Didn't notice that there was a local array that was so large. Seems like in summary, the PROGMEM was working, we just had a local array overrunning the available memory.

The float version I made also got the correct answer (18.xx). I'm assuming that you were trying to figure out the index stuff, so it was reading off the incorrect coefficient.

At some point, I think I need to find time to go through the code and make it more embedded friendly; I don't like large local variables. I also wonder if more pre-computing can be done to speed everything up. But, it is feasible to run on Teensy, so that's great!
 
brtaylor said:
Thanks Mike! Didn't notice that there was a local array that was so large. Seems like in summary, the PROGMEM was working, we just had a local array overrunning the available memory.

The float version I made also got the correct answer (18.xx). I'm assuming that you were trying to figure out the index stuff, so it was reading off the incorrect coefficient.

At some point, I think I need to find time to go through the code and make it more embedded friendly; I don't like large local variables. I also wonder if more pre-computing can be done to speed everything up. But, it is feasible to run on Teensy, so that's great!
Click to expand...

Glad you got it working. And yes was experimenting alot before I hit on the idea of changing everything to floats. Keep us posted :)
 
mjs513 said:
...
And yes I am a glutton for punishment
Click to expand...

This seems to be a recurring thing :) Good work!

Just clicked 'EGM96 geoid model' library link for the ReadMe - interesting the GPS sphere assumption doesn't fit the 'round' earth theory :)
 
Bummer the CrashReport didn't help the issue. Wondering if it captured something? Was there an 8 second pause before restart?

Maybe - if so - the if (CrashReport) could have printed with a delay(10000); after the printing? That would have kept it from printing and quickly crashing again before USB could transmit the details?
 
defragster said:
Bummer the CrashReport didn't help the issue. Wondering if it captured something? Was there an 8 second pause before restart?

Maybe - if so - the if (CrashReport) could have printed with a delay(10000); after the printing? That would have kept it from printing and quickly crashing again before USB could transmit the details?
Click to expand...

Morning Tim

Unfortunately no 8 second delay. The minute I opened the serial monitor it would go into a infinite restart loop. The delay might have worked can't remember if i tried that - probably not.
 
Morning all,

Several Months ago, I was having a strange crash, turned out a total memory corruption issue, that the normal CrashReport could not report it.

I resorted to augmenting unused_interrupt_vector in startup.c to print out most of the same stuff that CrashReport would output. using printf_debug(...)

I turned on Debug output in the kernel:
Edited teesny4\debug\printf.h and uncommented: //#define PRINT_DEBUG_STUFF

Which outputs by default to Serial4 at 115200
...
 
KurtE said:
Morning all,

Several Months ago, I was having a strange crash, turned out a total memory corruption issue, that the normal CrashReport could not report it.

I resorted to augmenting unused_interrupt_vector in startup.c to print out most of the same stuff that CrashReport would output. using printf_debug(...)

I turned on Debug output in the kernel:
Edited teesny4\debug\printf.h and uncommented: //#define PRINT_DEBUG_STUFF

Which outputs by default to Serial4 at 115200
...
Click to expand...

Think that is what was happening in this case as well. Even when I went back this morning and added the delay(10000) it CrashReport didnt report the error. However, had a feeling it was a memory issue but just was sure where until I played a bit more.
 
How large is: _coeffs?

Is it: #define _coeffs (65341) //!< Size of correction and harmonic coefficients arrays (361*181)

If so: This line: double p[_coeffs+1], sinml[_361+1], cosml[_361+1], rleg[_361+1];
is trying to use at least: 531424 of stack space, which is unlikely to work? about 518K probably wont fit into DTCM.

It probably also won't fit fully into RAM2 either although maybe if the only p was placed there and the others stayed in DTCM.
 
Yep - found that out when I was experimenting on where to put that "p" array. Thats why I winded up putting it EXTMEM and made it global and using a T4.1:
Code: 
EXTMEM p[_coeffs]/CODE]

Changing the doubles to floats does work though. :)
 
You must log in or register to reply here.
Back
Top