Only compile specific Audio-lib files included

Status
Not open for further replies.

manicksan

Well-known member
I have a question

Is there any way to just compile the files needed?

I tried to remove all
these includes:
//#include <Audio.h>
//#include <Wire.h>
//#include <SPI.h>
//#include <SD.h>
//#include <SerialFlash.h>

and only included the specific needed

but it still compiles all files in the Audio-lib Folder

I have no problem with it doing a full recompile,
but find it very unnecessary to compile files not used.
 
Take a look at Audio.h.
If you want to save a few seconds more, try GCC10
wink.png
It think it compiles faster, but did not measure it. On the other hand, that actual "preview" by ARM produces defective code with "fastest" setting (not even "blink" runs). Other optmization levels are OK.

But be happy.. I have a ESP32-Project that compiles 3 Minutes.... and after that, takes ages to upload over the slow serial..
 
Last edited:
I did not use Audio.h at all
and I only included the needed files,
but still it compiles all C files in the audio folder,
I can see that by enabling the compile log output.

I'm not talking about some seconds, it takes 40 seconds
to compile a small test program for teensy 4.0
and most of that time is spent on every c file that is not actually
needed for the program.

Normally I'm using VSCODE with PlatformIO
where it saves the output files, and also the compile time is
much faster.

Right now I'm developing some plugins for the Arduino IDE
and want to have so little compile time as possible.
It's not really necessary but I think there is a lot of people out there
that also thinks it takes forever to compile right after the IDE
has started.

Yes programming the ESP via serial is very slow even at 115200bps
Haven't you considered using OTA programming, that is a lot faster.

Do you have an SSD that is much faster when doing a lot of compiles.

It makes me thinking about using a RAM disk to make it fast as possible,
it could be something for you to, just to store the libraries and such stuff,
the actual code one write is sometimes not that big, so it could be an option.
 
I'm not talking about some seconds, it takes 40 seconds
to compile a small test program for teensy 4.0

Maybe Windows Defender is slowing things down?

FWIW, using the Arduino IDE on my Linux desktop, the WavFilePlayer example compiles all files in 12 seconds. Admittedly, my desktop machine has fast hardware: Intel i7-8086K, Intel 905P SSD, and 64GB RAM.

Not sure how much of that difference is Windows vs Linux. But we do know for sure Windows Defender and other anti-malware causes a huge slow-down on many Windows machines. Look into that first!
 
I have an older AMD Ryzon 5 3.6GHz with 16GB RAM and fast M.2 SSD
Virus-Scan disabled for Arduino temp-dirs, Teensy core and lib folders.

Its more than fast enough for everything but Arduino :)
 
I'm curious, why is it recompiling all kinds of things that didn't change? It happens for me too (Linux, just by hitting verify twice).
 
It is really compiling?

The Arduino IDE first runs the compiler many times with -E, which means to just run the preprocessor but not actually compile any code. It does this to discover which include files you have used. Early versions of the Arduino IDE tried to do this by just parsing the code. But that way can't take into account #if and #ifdef which may or may not cause some includes to be used.

If you look casually at the verbose output, those many times with -E might be mistaken for compiling code.

Then it actually does run the compiler the normal way. That is where you should see it reuse compiled code.
 
Maybe Windows Defender is slowing things down?

FWIW, using the Arduino IDE on my Linux desktop, the WavFilePlayer example compiles all files in 12 seconds. Admittedly, my desktop machine has fast hardware: Intel i7-8086K, Intel 905P SSD, and 64GB RAM.

Not sure how much of that difference is Windows vs Linux. But we do know for sure Windows Defender and other anti-malware causes a huge slow-down on many Windows machines. Look into that first!

Windows box here 2 generations back and only 4 not 6 core i7-6700 showing 3.4 GHz base speed not 4.0 GHz - its turbo Max is 4.0 GHz and i7-8086 is 5 GHz.

WavFilePlayer takes 35 seconds initial and 6 seconds for recompile without change.

6 second recompile says it is making good use of prior compiled files.

This is running all { IDE and build TEMP data } from (non boot) SSD with all protection scans disabled on that drive. It brings all 4 ( *2 HypeThreads ) to life and memory isn't a factor - it has 32GB but doesn't blip over 8 GB used from start. System doesn't seem to make best use of SSD and it is on SATA port not a faster direct MOBO connect.
 
Indeed, most of it is with -E. The sketch itself is recompiled and then everything is relinked. It's quick, so not an issue - but I expected it to do nothing (since no files have been touched). Maybe it's the price for arduino features.
 
Now I have compiled the same project

First Arduino IDE (~40sec)

And the the same "sketch" imported as PlatformIO project (14.4sec)

the output logs are in the attachments

So there you have it, platformio is ~3 times faster + that you can do a clean to ensure it's a full recompile it you encounter any problems.

Hopefully when Arduino IDE PRO is completed and working
it can use the same tech because it's based on VSCODE (compiled typescript running in Node.js),
so it will be the same speed as platformio.


Also I tried to copy and run the Arduino IDE install to the same drive as PlatformIO is on
but that resulted in 45sec time instead.
 

Attachments

  • ArduinoIDE_compile.zip
    8.1 KB · Views: 48
  • VSCODE_PlatformIO_compile.txt
    13.2 KB · Views: 61
found a solution

by adding build.path to preferences.txt (in windows located at %userdir%\AppData\Arduino15)

example that creates a folder inside Arduino Install dir.
build.path=.\buildTemp

this will ensure that the build output is not deleted when the IDE closes (checked the Arduino src code):
Code:
 /**
   * Gets the build path for this sketch. The first time this is called,
   * a build path is generated and created and the same path is returned
   * on all subsequent calls.
   *
   * This takes into account the build.path preference. If it is set,
   * that path is always returned, and the directory is *not* deleted on
   * shutdown. If the preference is not set, a random pathname in a
   * temporary directory is generated, which is automatically deleted on
   * shutdown.
   */
  public File getBuildPath() throws IOException {
    if (buildPath == null) {
      if (PreferencesData.get("build.path") != null) {
        buildPath = BaseNoGui.absoluteFile(PreferencesData.get("build.path"));
        Files.createDirectories(buildPath.toPath());
      } else {
        buildPath = FileUtils.createTempFolder("arduino_build_");
        DeleteFilesOnShutdown.add(buildPath);
      }
    }

so the next time arduino is started and a compile is done
it reuses the prev. compiled files.
 
Great find and just tried it.
for me the path to preferences was "%userdir%\AppData\local\Arduino15"

Now we need a simpler way to force recompile, than to delete by hand the files or change Teensy tools entries
 
Great find and just tried it.
for me the path to preferences was "%userdir%\AppData\local\Arduino15"

Now we need a simpler way to force recompile, than to delete by hand the files or change Teensy tools entries

Yes. I asked for this long time ago :)
 
I have tried to compile a similar program on:

At a VM with Catalina (only used for testing my new extensions) the compile time is much faster.
If I look at the compile output it have the similar functionality as the platformio version.

On Ubuntu 20.04 also on a VM the full compile for above program is ~25sec.


Anyway I have created a tiny plugin/extension (6.38KB)

This Extension Make It possible to have the compiler output the files
to the sketch subfolder called build.

This make it possible to save the build output for every sketch,
so that recompile is faster for all already compiled sketches.

This extension also have a Clear Build that is working on both the sketch based build output
and the standard temp folder/(the folder selected in preferences.txt)

it's available at:
https://github.com/manicken/arduinoIDEsketchBuildPath

/Jannik
 
Two questions.

Who is responsible for the
teensy "platform io" implementation?

As the compiler for the "platform io" is much faster,
and the output(compiler log) is much cleaner and includes a nice tree view(probably just the preprocessing).

Why aren't that used for the Arduino IDE instead?
 
By the way the cost of recompiling all the files is considerably greater with (spinning) harddrives than with solid
state drives - IMO the best way to get responsive software is to force developers to work with slow hardware
so they actual notice the bottlenecks!
My backup machine is old enough to have hard discs and is an order of magnitude slower to compile audio lib
programs.

Thinking about this, has anyone managed to configure Arduino/Teensyduino build directory on a RAM drive?
I found this relevant bug report: https://github.com/arduino/Arduino/issues/8812

[ Just noticed this point was already made in the thread earlier... ]
 
Status
Not open for further replies.
Back
Top