How do you organize large sketches?

KrisKasprzak

KrisKasprzak

Well-known member
All,

I have a few "large" .ino files, one program is around 8500 lines, it has some 100 functions, and 20 libraries. I'm familiar with where things are in the code and search is helpful, but digging through 8500 lines is a pain. Some areas of code are edited rarely but don't warrant creating a .h (interface creation code to create buttons for example). I'm using Arudino IDE only.

I've been considering moving "less" frequently edited code to separate .ino files but keeping them in the same folder, that way I can edit if needed. I'm not sure this is good or a safe practice.

What do others do when your code grows in line count? Any suggest greatly appreciated.
 
The latest version of my TeensyMIDIPolySynth includes a Teensy 4.1 (which manages all MIDI interfaces, 7" TFT touchscreen display, EEPROM storage/retrieval of 16 configurations, & a 500kbps serial interface shared with the audio T4.0) & a Teensy 4.0 (which manages an Audio Interface Adapter, & a 500kbps serial interface shared with the display T4.1). The source is contained in a separate source file for each of the two Teeny processors as follows: T4.1: 49721 SLOC, & T4.0: 20262 SLOC. Looking across all of my Teensy projects, these are, by far, the largest Teensy source files that I have worked with. Like you, I do all of my development in the Arduino IDE (primarily 1.8.19+TD1.59, but I have also played with 2.3.2). I always organize the functions in my source files in alphabetical order, and (as my one key point to offer) I include function headers for each function at the top of the file for easy reference. As inconvenient as this (such large, single files) may appear to some, I have not had any problems developing/modifying/maintaining my source this way, but this particular approach may not be for everyone. As a lifelong software developer (at work), I wrote & worked with source in C that numbered in the hundreds of thousands of lines, and that was definitely logically divided into well organized "modules". With the way that the Arduino IDE manages many aspects of the "ino" files (e.g. simplification for new users, using the setup() & loop() paradigm), I have not been brave enough to attempt the same kind of "organizing" approach in my Teensy development activities. Sorry, I don't really have any "magic bullets" to offer (other than the suggestion for including function headers at the top of the source files for quick reference already mentioned).

Mark J Culross
KD5RXT
 
When I have code that gets too large, usually I look for the most mature and unlikely to change portions. That's the stuff I tend to put into another file or create a library if I'm sure it'll get reused between programs.
 
I always tend to keep the *.ino file as short as possible, and put all my other stuff into .cpp with associated .h files within a src sub directory. Arduino IDE 2. has no issues with this and by not using multiple ino files, I avoid surprises due to Arduino preprocessor.
I also tend to code the ino as it were a cpp file so that non-Arduino compilation (i.e. Makefile based) is possible.
 
WMXZ said:
I always tend to keep the *.ino file as short as possible, and put all my other stuff into .cpp with associated .h files within a src sub directory.
Click to expand...
This tends to be my approach too.

e.g. I recently created a simple serial logger for listening in and logging the data in each direction over a serial link. So two ports per link, one for each direction. There is then a text setup file on the SD card to set the baud rates and a few other config options.

Total project is around 1000 lines so manageable in a single file.

I created a class called PortPair that handles everything to do with the logging (configure ports, open/close files etc...)
I created a class called Config that reads the text config file and parses it into some data structures.

Both of those classes each have a .cpp and a .h file.
There is then a similar structure used for logging CAN packets.

The main .ino is then the bare minimum to join these blocks together. Read the config, configure the ports and then loop reporting status and checking for the start/stop logging button.

The longest file is the config file stuff at around 300 lines, it's all the printf / scanf lines needed for parsing setup files, reporting settings, outputting example settings files etc... so long but nothing technically complicated.
This way if I want anything to do with the serial logging I know it will be in a specific far smaller file. I also means I don't need to worry so much about accidentally reusing the same variable name for say tracking CAN logging status and serial logging status, those variables are only visible within their class.

If you don't like the whole c++ classes approach you can still split things this way using .c and .h files.
 
AndyA said:
If you don't like the whole c++ classes approach you can still split things this way using .c and .h files.
Click to expand...
You can also use namespaces which makes grouping and naming functions easier
(note that you cannot make variables and functions private as when using classes)

in SomeName.h file:
C++: 
#pragma once 
namespace SomeName
{
    void foo()
    {
        
    }
}
note pragma once is the same as using the include guard statements:
C++: 
#ifndef SOMENAME_H
#define SOMENAME_H
// code
#endif
which should be used if the code is used by another two or more include files

in main "sketch" file
C++: 
#include "SomeName.h"

void setup()
{
    SomeName::foo();
}
void loop()
{
    
}
 
manicksan said:
You can also use namespaces which makes grouping and naming functions easier
Click to expand...
The main issue I have with this approach is that if I have two .cpp files (or a .ino and a .cpp) that both include SomeName.h then you will get linker errors if the namespace contains anything other than constants or inline functions.
 
@ AndyA
I did also have problems with linker error while trying to use namespaces (note when using classes it's also the same problem).

consider the following c++/arduino files:

SketchName.ino:
C++: 
#include "Test2.h"
#include "Test3.h"

void setup() {
  // put your setup code here, to run once:
  Func1();
  Func2();
  NS3::Func3();
}

void loop() {
  // put your main code here, to run repeatedly:

}

Test1.ino:
C++: 
#include "Test3.h"

void Func1() // function definition
{

}

Test2.h:
C++: 
#pragma once

void Func2(); // function declaration

Test2.cpp:
C++: 
#include "Test2.h"
#include "Test3.h"

void Func2() // function definition
{

}

Test3.h:
C++: 
#pragma once

namespace NS3
{
  void Func3() // function definition
  {

  }
}
then I asked chat gpt the following:
Code: 
why cannot i include Test3 many times even tho i use #pragma once

what are the reasons c++ compiler ignore #pragma once

the answer is the following:
Code: 
The issue you're encountering is related to the way #pragma once and include guards work,
particularly in the context of defining functions within a header file.
Here’s a breakdown of why including Test3.h multiple times causes problems and the solutions to address this:

Issue Explanation
Function Definitions in Header Files:

In your Test3.h file, the function Func3 is defined within the header.
This means that every time Test3.h is included, a definition of Func3 is provided.
When the compiler encounters multiple definitions of the same function (in different translation units that get linked together),
it raises a multiple definition error.

#pragma once and Include Guards:

#pragma once ensures that the header file is included only once per translation unit (source file including it directly or indirectly).
However, this does not prevent multiple definitions across different translation units (e.g., main.ino, Test1.ino, and Test2.cpp all including Test3.h).

Correcting the Issue
To correct this issue, function definitions should be moved to a source file (.cpp or .ino file), while only declarations should remain in the header file. Here’s how to refactor your files:

Move Function Definition to a Source File:

Move the definition of Func3 from Test3.h to a new file, say Test3.cpp.
Update the Header File:

Ensure the header file (Test3.h) only contains the declaration of Func3.

So this is the correct way of having Test3.h and Test3.cpp
(note that i found out that you can actually have "private" functions inside a 'namespace')
Test3.h:
C++: 
#include "Test3.h"

namespace NS3
{
  void Func3(); // function declaration
}

Test3.cpp
C++: 
#include "Test3.h"

// private functions here (only visible in this file)
void Func3private()
{
}

namespace NS3
{
  void Func3() // function definition
  {
    Func3private(); // call 'private' function here
  }
}

if you however prefer it, you can use classes like this:
Test4.h:
C++: 
#pragma once

class Class4
{
 public:
  void Func4(); // function declaration
};
// short instance name of Class4 which can then be used with 'normal' dot notation: C4.Func4();
extern Class4 C4;
// this is what most 'easy to use' arduino libraries do
// for example Serial is actually a instance of usb_serial_class like this:
// usb_serial_class Serial;

Test4.cpp
C++: 
#include "Test4.h"

void Class4::Func4() // function definition
{

}
Class4 C4; // same comment as above

sure it's many files,
but it's much easier to maintain.

preferable I also want to have minimal code in the main.ino/cpp file
to make sure that it's easier to understand the functionality
(specially if you leave the project dormant for a while, and want to update it later)
 
Last edited:
WMXZ said:
I always tend to keep the *.ino file as short as possible, and put all my other stuff into .cpp with associated .h files within a src sub directory.
Click to expand...
I have been doing this for some time, but without the src sub directory. It requires some discipline / care with includes and externs, but I've found it much easier to maintain stuff, and results in a set of code "modules" (.cpp/.h pairs) that are easily re-usable in other sketches.

Curious, what is the purpose/benefit of putting .cpp and .h in a src sub directory?
 
ninja2 said:
Curious, what is the purpose/benefit of putting .cpp and .h in a src sub directory?
Click to expand...
Mostly an extra level of organisation and structure.
The arduino build system assumes all your code is either in the top level directory or the src sub directory. But within the src subdirectory you can have any structure you want.

e.g. for one project I have a src/hardware directory which then contains all the files related to talking to the hardware (one pair per type of device), a src/network which contains all the network related code (one pair for UDP, one for TCP, one for HTTP) etc...
 
So are you saying that Arduino will pick up everything in the src directory and it's child directories?
 
BriComp said:
So are you saying that Arduino will pick up everything in the src directory and it's child directories?
Click to expand...
Yes.
When you perform a build in arduino the entire src directory, including all subdirectories and their contents, is copied to the temporary directory that it then uses to perform the build.
 
@ Arduino IDE 2 there is a setting "Show files inside Sketches"
(note the sketch project need to be in the "Sketchbook location"@settings for this to work)

which enables the following:
Arduino2sketchContents.png

note that you can close files that have been opened from the src folder

so i would say that the preferable way is to only have,
the main sketch in the "sketch-root" folder and all other files in the 'src subdir'/'sub folders of src',

this way it wont matter how many files you have,
as you can close the ones that you are not currently editing


a last note, all above is not possible using the 'old' Arduino IDE
only putting files in src folder but you have to view them in another editor
 
a last note, all above is not possible using the 'old' Arduino IDE
only putting files in src folder but you have to view them in another editor
Click to expand...
If you use VisualMicro for VisualStudio (Comunity version) this is not a problem. You can open any number of files from any number of directories and work on them all.
 
manicksan said:
You can also use namespaces which makes grouping and naming functions easier
(note that you cannot make variables and functions private as when using classes)
Click to expand...
Actually you can by using anonymous namespaces. Here an example:

test.h
C++: 
#pragma once

namespace someNamespace
{
    extern int getVar();
    extern int publicVar;
}

test.cpp
C++: 
namespace someNamespace
{
    namespace // anonymous/private namespace
    {
        int privateVar = 42;
    }

    int getVar()
    {
        return privateVar;
    }

    int publicVar = 0;
}

Some ino:
C++: 
#include "test.h"
//...
//someNamespace::privateVar = 12; not possible, lives in anonyomous namespace
someNamespace::publicVar = 17; 
Serial.println(someNamespace::getVar());  // access private var with getter
//...
 
AndyA said:
Yes.
When you perform a build in arduino the entire src directory, including all subdirectories and their contents, is copied to the temporary directory that it then uses to perform the build.
Click to expand...
IDE 2.x allows you to get away with murder but you will eventually get caught if you're not carefull. I have gotten used to using python and swift where .h files are not used. If you write perfect code IDE 2.x will just piece all the .ino files together no matter if no function prototypes, or where global variables are defined, as long as loop() and setup() are in the .ino with the name of the enclosing top folder.

However, as soon as the compiler hits an error it will stop finding functions and globals seemingly randomly. You will have to add a function prototype for whatever the compiler is asking for, then recompile ect. until it gets to the real bug or sometimes its easier to just undo whatever you just added, although depending on the cache state that might not work.

Looking at my last few major programs I typically have no .h files, 6 to 16 .ino files but call all kinds of .h from libraries. Not suggesting you do this but IDE 2 will let you.
 
Last edited:
bicycleguy said:
However, as soon as the compiler hits an error it will stop finding functions and globals seemingly randomly. You will have to add a function prototype for whatever the compiler is asking for, then recompile
Click to expand...
That happens occassionally with IDE 2 even if \scr is not in use.
 
Hi,

For big Arduino projects, break your code into separate .ino files by function, like one for sensors and another for the user interface. Create libraries for reusable code, use clear names, and add plenty of comments. This makes your project easier to handle, edit, and update later.

Jack ;)
 
I use VSCode with the platformio extension, use the object-oriented paradigm (even for classes that I know will have only 1 instance) and modern c++ when libraries allow it (so, not a lot), and have my own private repository to put classes that I reuse between teensy projects.
This allows me to work with teensy projects in the same environment and using the same tools (mostly in the form of vscode extensions) as my other development projects, which imho increases comfort and productivity vs the Arduino IDE (mostly true for people who already have other development projects and don't really want a separate IDE for each)
 
JackJohnstone said:
Hi,

For big Arduino projects, break your code into separate .ino files by function, like one for sensors and another for the user interface. Create libraries for reusable code, use clear names, and add plenty of comments. This makes your project easier to handle, edit, and update later.

Jack ;)
Click to expand...

I wouldn’t use multiple .ino files in your project; that sounds strange to me. Have one .ino file for your “main code” and then the rest headers and C++ or C sources.
 
Well I've been messing around with multiple files resulting with mixed emotions. Love the ability to organize, but search seems to not cross over multiple files. Since the code in my other files is not edited often, this is only a minor nuisance.

Not sure if I will continue multiple files.

Thanks for all the great ideas.
 
You must log in or register to reply here.
Back
Top