Arduino 1.6.3 & Teensduino 1.2.3 = too many files generated during compilation

V

vsurducan

Member
Hi all,

I'm puzzled about the huge numbers of files produced by arduino/teensduino during compilation. All those files are written on a temp folder, consuming time and slowing down the compilation process.
Even not used examples under the teensduino are always processed. Why?

For example my sketch has 8 files all of 20k, but the compilation has 97 files in 1800k...
There is a way to drop those silly files without reinstall the teensduino without examples?

thx,
Vasile
 
vsurducan said:
There is a way to drop those silly files without reinstall the teensduino without examples?
Click to expand...

No, not really.

Those files may seem silly, but they give you things like pinMode(), digitalWrite() and Serial.print(). Even though you write only a small sketch, Teensy is implementing the complex USB protocols you select in Tools > USB Type.

All those files are written on a temp folder, consuming time and slowing down the compilation process.
Click to expand...

Each time you recompile, it's supposed to reuse the compiled code from last time, if the source hasn't changed. It's supposed to only take longer the first time you compile, or after you've made some sort of configuration change.
 
PaulStoffregen said:
No, not really.

Those files may seem silly, but they give you things like pinMode(), digitalWrite() and Serial.print(). Even though you write only a small sketch, Teensy is implementing the complex USB protocols you select in Tools > USB Type.



Each time you recompile, it's supposed to reuse the compiled code from last time, if the source hasn't changed. It's supposed to only take longer the first time you compile, or after you've made some sort of configuration change.
Click to expand...

OK, but why I can see files such as keyboard, joystick, midi, etc. since I do not use them in my sketch? Those are examples.
If you open two sketches and compile them consequently, you will see that a new temp folder is generated after every compilation. Is so funny to have 1.8M of compiled files and 40 seconds of compilation for a source code of 20k isn't it ?

I've reported this problem to Arduino forum but nobody cares. At least in the teensduino, let the user to deactivate all the examples...

thx,
Vasile
 
vsurducan said:
Those are examples.
Click to expand...

No, you've misunderstood. They're part of the core library which implements the USB stack. They're not merely examples.

Indeed many of those files have lots of code that isn't compiled, do to #ifdef directives. But the core library gets configured in many different ways using conditional compilation.

There really are some very good reasons why the core library source code is structured this way. The compiler generated files have lots of extra debug data built in by the compiler.

Is so funny to have 1.8M of compiled files and 40 seconds of compilation for a source code of 20k isn't it ?
Click to expand...

Have you measured the time difference between the first time you compile, and the second compile where you've only changed things in your sketch?

In these modern times, 1.8 megabyte of short-lived temporary on a PC hard drive isn't worth worrying about.
 
Hi Paul,
Not the length of the temp folder is the real issue, (which btw must be deleted by hand) but the compiling time caused by multiple files/fragmentation written on the HDD. It's huge compared with any decent compiler. I remember the old days when compiling the code for a PIC microcontroller on a 486 machine took 20 seconds and it was too long.
Now, with an I7 four core 2.4GHz machine and a SSD it still take 30s the first compilation of a blank sketch for the Arduino or Teensy, just because the developers does not care about this.

Please compile a blank sketch:

void setup() {
// put your setup code here, to run once:

}

void loop() {
// put your main code here, to run repeatedly:

}

Then inspect your temporary folder and see what you get: 90 files

This is a good example of how should not be parsed a source file by any compiler or IDE/compiler combination.

Please let me know what compiler should I use for teensy 3.1 instead of Arduino/Teensduino IDE. This combination is not for me and it should not be (as is) for any beginner.

thx,
Vasile
 
Last edited:
On my PC (admittedly a 6 core I7), the first verify takes approx 5 seconds. Subsequent runs are too quick to time by a stopwatch, just a fraction of a second.
 
My late-2011 Macbook with 4 core I7 takes about 4 seconds on the first compile, and subsequent compiles are only a fraction of a second.
 
It takes 20 seconds on a 4 year old toshiba laptop.
Actually I'd say that this latest version is faster than before, especially when recompiling.
 
Arduino compiles all sources every time (with some exceptions, e.g., CPU core .a's) - first time for a project. Thereafter it's supposed to be only changed sources.
On my SSD-based I5, a build is 1-3 seconds.
 
Twenty seconds to do a build, oh the horror.

I currently have 1/2 hour build times for my day job. This is down from 2-3 hours on previous machines that were slower. When I want to do a full spec run, it is something like 18 hours on one machine, and 9 hours on the new machine. I recall consulting with customers who if there were to rebuild all of their software, it would be a week.

Off hand, however, I would say your machine is misconfigured. Or you don't have enough main memory on the system, and it is paging itself to death.
 
Actually, this thread has got me thinking about a couple crazy ideas to (maybe) store pre-compiled copies of the core library for all possible permutations of the various -D{define} settings.

Many times I've also considered hacks in the IDE to run more than one instance of the compiler at a time. But that idea worries me, because the people who report long waits clearly have some terrible problem with their computers. A quad core Intel I7 chip with SSD storage should *not* take 40 seconds to compile a simple sketch. The compiler uses quite a lot of memory, so running more than one instance at a time could make such broken computers go from bad to much, much worse, even if they have 4 CPU cores.

One idea that is absolutely NOT up for consideration is removing any features from the core library. There's a lot of stuff in there, and indeed very few people use even 20% of it, but everyone uses different parts. It's all required for good compatibility with the huge universe of Arduino sketches and libraries!

I might consider merging some of the files together, at some point. But that has to be done very carefully, because there are complex issues with the linker bringing in weak linked interrupt routines based on whether they're in the same compile units. Changes that would increase compiled code size for people's projects are also off the table.
 
PaulStoffregen said:
On my PC (admittedly a 6 core I7), the first verify takes approx 5 seconds. Subsequent runs are too quick to time by a stopwatch, just a fraction of a second.
Click to expand...

Hmm, the Arduino IDE is using for real the multicore processing? I guess not.
Did you install the IDE in the program files (using the exe pack) or in the documents folder (using the zipped pack) ?

Actually I'm playing with this (funny) IDE on three computers: a single core 3GHz with PATA 5400 rpm HDD WINXPSP2, a four core I7x2.4GHz with SATA SSD WIN7, and and atom 1GHz computer with PATA 5400 rpm HDD, WIN7 and differences aren't spectacular.

Indeed the blank sketch does not take longer than 5sec on I7.

The six files sketch took as I said, 25-30 seconds for the first compilation on I7 and around 50-60s on the 3GHz single core machine.


There is a chance to write those files on a temporary virtual drive defined in the computer's memory?

thx,
 
Last edited:
MichaelMeissner said:
Twenty seconds to do a build, oh the horror.

I currently have 1/2 hour build times for my day job. This is down from 2-3 hours on previous machines that were slower. When I want to do a full spec run, it is something like 18 hours on one machine, and 9 hours on the new machine. I recall consulting with customers who if there were to rebuild all of their software, it would be a week.

Off hand, however, I would say your machine is misconfigured. Or you don't have enough main memory on the system, and it is paging itself to death.
Click to expand...

Michael, the system has 8G RAM and fresh win installed on it.
So, where can be the problem?
 
Pedvide said:
It takes 20 seconds on a 4 year old toshiba laptop.
Actually I'd say that this latest version is faster than before, especially when recompiling.
Click to expand...

At recompiling is quite ok, unless you're not compiling two different sketches one after the other...
 
vsurducan said:
Indeed the blank sketch does not take longer than 5sec on I7.

The six files sketch took as I said, 25-30 seconds for the first compilation on I7 and around 50-60s on the 3GHz single core machine.
Click to expand...

If your 6 file sketch takes a very long time, but a blank one is fast, I believe that would be a issue or bug, where the "forum rule" applies. I can't do anything to investigate problems which I can't reproduce here. As a general rule, you have to post sample code that reproduces the problem if you want me to investigate.

Did you install the IDE in the program files (using the exe pack) or in the documents folder (using the zipped pack) ?
Click to expand...

I tested on Linux 64 bit and Macintosh, not Windows.
 
vsurducan said:
Michael, the system has 8G RAM and fresh win installed on it.
So, where can be the problem?
Click to expand...
I've seen this problem on the past with systems with inadequate main memory. 8G should be more than enough. You need to ask experts on your OS what are the ways to use memory efficiently. Under Linux, I would run the top application to find memory hogs that might not allow other programs to get their fair share of memory.

I don't use Windows at all (except to install new firmware on my camera), so I can't help you there.
 
Well, I don't know guys how it works for you in 3seconds ( I'm talking about the first compilation or the first verification of a sketch and not about something else), but I'm glad to hear someone is happy with the Arduino IDE.
:)
Definitely not the computer or the Win is the problem but the IDE

The speed of the process is lowered mostly by the multiple writings on the SSD (HDD) during the first compilation and everytime something has been changed in the board parameters...

thx for your comments,
Vasile
 
Last edited:
vsurducan said:
Definitely not the computer or the Win is the problem but the IDE
Click to expand...

In my almost 30 years of experience as an engineer I have learned that most of the time the problem is not the computer or the software.
In the vast majority of cases the problem is sitting on the other side of the keyboard :)
 
Headroom said:
In my almost 30 years of experience as an engineer I have learned that most of the time the problem is not the computer or the software.
In the vast majority of cases the problem is sitting on the other side of the keyboard :)
Click to expand...

Good, with yours 30 and my 30, I know you might have right. :) Arduino IDE is a big crap anyway...
best wihes,
Vasile
 
Maybe it helps to disable the virusscan on the arduino and temp folder for some fileextensions (c, cpp, h, o... etc) ?
 
Just measured it:

First compile empty sketch: 12 sec, after that less than 1 sec.

SSD: Samsung SSD 850 EVO 250GB
System : i5 3570K @ 3.4 GHz, Win 7 Ultimate, 8 GB


and ...let me add another 30 years :)
 
I use the Arduino IDE on different OSs on a mid 2010 iMac 4-core i7. I use it on OS X, Linux and Windows 7 with the Linux and Windows 7 installations running in the VM Fusion virtual machine.

I don't have such long compilation times on any of the various installations and they are more in line what has been posted by other users. I know that this does not help figuring out what the problem is but it certainly provides another hint that the IDE might not be the problem.

The quickest compilation times I actually don't have with the Arduino IDE but with the Arduino Eclipse IDE. With the Teesies the difference is really not that different, but for my base Arduino UNO that I keep around for testing if libraries run on it the compilation and particularly the upload times are worlds apart.
 
Frank B said:
Just measured it:

First compile empty sketch: 12 sec, after that less than 1 sec.

SSD: Samsung SSD 850 EVO 250GB
System : i5 3570K @ 3.4 GHz, Win 7 Ultimate, 8 GB


and ...let me add another 30 years :)
Click to expand...

Oh my!
Add at least another 20 for Paul and another 30 for Michael and we combine 140 years of engineering experience across different disciplines and we can't figure out this gnawing problem. What does that tell us about the current state of engineering ;-)
 
You must log in or register to reply here.
Back
Top