Adrift in a sea of #ifdefs

mborgerson

Well-known member
I have 4 different models of Teensy (3.2, 3.5, 3.6, 4.0) and they have all provided many hours of educational and interesting hardware and programming projects. There are lots of excellent libraries for the Teensy boards and I have learned a lot from examining their source code. However, when I read through the source I often get lost in the sea of #ifdef statements that allow the same source file to work with the various Teensy boards. It is troubling to be working on at T3.6 project and reading through a library source code file only to find an "#endif" statement at the bottom of the screen. I start to wonder if I have been reading through the source for at T3.2 or T4.0 and have to scroll back up to find the matching #if defined(Teensy_3_6).

This leads me to the following questions:

1. Is there a text processing tool that will allow me to sort through the #define statements in a source code file and eliminate all the source for processors other than the one I select?

2. What are the pros and cons of writing library code with one large source file using #ifdef to handle multiple processors or boards versus writing separate source code files for each processor (which will need far fewer #ifdef statements, but have more files to maintain)?

I guess I was spoiled by spending a few decades where I designed systems from the bottom up. I got to pick the processor, design the circuit, lay out the PCB, and write the firmware. About the only #ifdef statements in my code were "#ifdef DEBUGPRINT". As a result the customer got firmware source for the board in hand and didn't have to cope with lines of code designed for someone else's board.

What is the current preference of the computer science academic community on this topic? I've only had a few lower-division programming courses (back in the late 1960's) so I'm a bit out of touch with academic opinion. Being self taught can go a long ways if you pick a narrow enough field of endeavor, but it can lead to a bit of tunnel vision.
 
1. Is there a text processing tool that will allow me to sort through the #define statements in a source code file and eliminate all the source for processors other than the one I select?
I've used an awk scriptlet to do this. While you can use the C preprocessor (avr-cpp in Arduino, arm-none-eabi-cpp in Teensyduino) to do all preprocessing, it is easier to write a simple awk script to only process specific preprocessor directives it knows about; it's simpler, too. It would be easy to write one in Python3 too, and Python 3 is probably easier to install than awk (there should be ready-made installers for all OSes; in Linux, all distros have it in their package management).

As I'm a Linux user (haven't used Windows in well over a decade), with a POSIXy worldview and background in scientific computing, and am only a hobbyist wrt. microcontrollers and embedded systems, I won't bother you with my opinions on your other questions.
 
This is exactly why I refactored my Snooze library into different HALs for the different Teensies even if 90% of the code base is the same. It was getting to the point that I couldn't follow my own code anymore before refactoring.
 
1. Is there a text processing tool that will allow me to sort through the #define statements in a source code file and eliminate all the source for processors other than the one I select?

Instead of manipulating the source file you might also consider a state of the art development system. Here e.g. how vsCode displays a part of core_pins.h. It dimms the "undefined" parts of the code which makes browsing such code much more convenient.

Anmerkung 2020-04-23 203529.jpg

I find it amazing how this and other (nowadays standard) helpers like code completion, error flagging, display of function parameters... can increase productivity. I'm sure Eclipse, Atom or Sublime have similar features.
 
A potential problem with vsCode and other such smart editors is that, for a Teensy library, the processor isn't defined in the library file, but in a board file that is selected by Arduino when you specify the board for which you are compiling. I must admit that I need to do a lot more research to find out where the #defines come from before I can start to work with tools that could slim down the source code---or it's display in an editor.
 
A potential problem with vsCode and other such smart editors is that, for a Teensy library, the processor isn't defined in the library file, but in a board file that is selected by Arduino when you specify the board for which you are compiling. I must admit that I need to do a lot more research to find out where the #defines come from before I can start to work with tools that could slim down the source code---or it's display in an editor.

Luni's support for VsCode takes care of letting it know the #define'd board to perform with the smart editor.

Once you hear it called "Ifdef Hell' - and see this kind of necessary evil - you know why it got that name. In the case of Teensy it started with the 3.0 fresh and new on the ARM platform. Then it got a big MCU boost going to the 3.1 - just more resources and 'minor changes' ... then the 3.2 which uses the same MCU and just redid the bootloader MCU and PCB's LDO. Then along comes the 3.5 and 3.6 needing some tweaks throughout - most handled uniformly in header files for extra pins and resources - but of course some parts of the Arduino facing code - like EEPROM need a special case for T_3.6 running over 120 MHz - that was localized to the added call being a NOP when not needed, other times new hardware adds new unique files - sometime only simple tweeks 'inline' with #ifdef prevents duplicating code to maintain twice later. Then comes the Teensy 4.0 - still ARM - with significant changes - some trivial #ifdef to work with tested code - other comes from the Teensy4 subtree ... and with the Teensy 4.1 - using the same MCU - even that code is getting 'retouched'.
 
for a Teensy library, the processor isn't defined in the library file, but in a board file

Extracting this information is exactly what VisualTeensy (https://github.com/luni64/VisualTeensy/wiki) or other tools automatically do for you. See for example the Compiling & Uploading section in the Teensy User WIKI for some of them. E.g., VisualTeensy reads the relevant files and settings and translates them to the format vsCode requires. It then copies everything to the project folder and you are ready to work, compile and upload using the native tools from vsCode. The other tools work similarly.

If you want to browse source files for understanding code or finding out how other people did similar things, very valuable features are "go to definition" "go to declaration" and "go to references". E.g. in vsCode, you place the cursor in a symbol, function call, class name etc, press F12 and the editor opens the correct source files right at the line where you find the information. For me this is so much more convenient and efficient than all the `grep`ing tricks we were used to in the last century...
 
Last edited:
mborgerson, I just want to say that I really appreciate the poetic title of your post. :D. Cheers.

Thanks for that. To twist another oft-used trope: You can take the programmer away from the sea, but you can't take the sea away from the programmer. Between my Navy years, my oceanography years, and decades sailing the Salish Sea, salt water is part of me.

Just want people to know that not all useful languages can be compiled!

to continue the discussion, think of what we would have if Shakespeare wrote:

#ifdef READER_ENGLISH
...
#endif

#ifdef READER_GERMAN
...
#endif

#ifdef READER_FRENCH
...
#endif

Thankfully, they decided to publish as separate works!
 
Indeed that is a great thread title!

A more modern example for a written work would be the last box opened with an 80 page manual - with a useless 4 page subset of what is needed for information … repeated in 20 different languages.

Luckily The ARM Teensys all use the same language.
 
In the ADc I abstracted most of the differences to two files: a settings file with what each board can do and a hardware access layer to handle the different atomic access styles.
In the code I have things like #ifdef HAS_FEATURE_X, where HAS_FEATURE_X is defined in the settings file if that board has that feature.
 
I normally like to keep out of the coding religious wars.

I have worked with people who like everything in one giant source file, to other groups who believe that every function/method should have it's own source file. So instead of being in a sea of #ifdefs you are in a Sea of files.

Left to my own with doing for my own self, I probably am more in the camp of too many #if statements, probably mainly because I can be lazy at times and also because I have had too many cases where I had the same code replicated but slightly changed for different hardware and then there is an error found in it, which is then fixed for that specific hardware, but one or more other devices are not updated...

Other times I try to refactor things and maybe make the code more table driven or try to setup to maybe reduce the needed #if...

Like Debug Print statement... Often times the code starts off with
#ifdef DEBUG_OUTPUT
Serial.print(...)
#endif

So sometimes at start instead have a #if that defines something like DBGPrintf, where if the DEBUG_OUTPUT is defined it does the Serial.print... If it is not defined it defines either an empty macro or a always inline method which does nothing...

But again there are a thousand ways to do things...

So I try to go by: When in Rome do as the Romans
 
I see KurtE's point about maintaining multiple files. We should all remember that these #ifdef issues apply primarily to library code and need not pop up in a user program for a particular processor. I also appreciate Pedvide's work on the ADC code. His transitions from #ifdef PROCESSOR_TYPE to #if HAS_TWO_ADCS does make it easier to go through the ADC source.

SIDEBAR: Just last night I got 2-channel synchronized collection working in my acoustic ping detector experiment. It cut the duration of my interrupt handler in half! That allows faster collection rates and better resolution in the correlation and detection routines. Next step is to use two microphones with appropriate phase delays to get better forward gain and reduced side lobe noise pickup.

I will try to remember that complaining about library source code is like looking a gift horse in the mouth and asking "Is there a veterinary dentist near here?" We all get the benefit of a lot of hard work by smart people in these libraries. If their coding style is different from my own, I'll do my best to cope. However, since I'm in my mid-70's, you can expect the grouchy old man to pop up once in a while.
 
Back
Top