Library priorities in arduino-builder

Status
Not open for further replies.
P

PDOS

Well-known member
I have a few duplicate libraries. Arduino-builder chooses among these libraries and sometimes the choice is seemingly counter-intuitive.
And so I have been puzzling over the library priorities exercised by arduino-builder, where there are duplicate libraries.

What are the rules?

I naively assumed that this was determined by the order of the -libraries flag, with the last specified library having precedence.

But no, this is not so. It is more complicated than that, as explained by issue #574 (https://github.com/arduino/arduino-cli/pull/574).

I am excerpting these rules below because I think they can be useful to many people. Over and above these rules, I wish that I could specify an overriding priority without resorting to the "#include" syntax.
Code: 
This PR changes again the lib priority selection to improve backward compatibility. Now the algorithm should be (hopefully) 100% compatible with legacy algorithm used in the arduino-builder.

The priority is determined by applying the following rules, one by one in this order, until a rule determine a winner:

A library that is architecture compatibile wins against a library that is not architecture compatible
A library that has better "name priority" wins (see below for details)
A library that is architecture optimized wins against a library that is vanilla
A library that has a better "location priority" wins (see below for details)
A library that has a name with a better score using "closest-match" algorithm wins
A library that has a name that comes first in alphanumeric order wins
Usually the first four rules are enough, the rule 5 is rarely applied and the rule 6 is even more rare. Anyway they are there to not leave the selection process undefined even in those extreme cases.

@per1234 could you check if this solves #572?
I think that this issue alone calls for another release of the Arduino IDE.

Details about rules:

A library is considered compatible with architecture X if the architecture field in library.properties:

- contains explicitly the architecure X
- contains the catch-all *
- is not specified at all.
- (see table below for an example)
- A library is considered optimized for architecture X only if the architecture field in library.properties contains explicitly the architecure X.

architecture field in library.properties	Compatible with avr	Optimized for avr
not specified	                                                   YES	NO
architectures=*	                                                   YES	NO
architectures=avr	                                           YES	YES
architectures=*,avr	                                           YES	YES
architectures=*,esp8266	                                   YES	NO
architectures=avr,esp8266	                                   YES	YES
architectures=samd	                                            NO	NO

The "name priority" is determined as follows (higher is better):

Priority	Rule	Example for Servo.h
5	The name match the include 100%	Servo
4	The name match the include 100% with a -master suffix	Servo-master
3	The name has a matching prefix	ServoWhatever
2	The name has a matching suffix	AwesomeServo
1	The name contains the include	AnAwesomeServoForWhatever

The "location priority" is determined as follows (higher is better):

Priority	Rule
4	The library is in the sketchbook
3	The library is bundled in the board platform/core
2	The library is bundled in the referenced board platform/core
1	The library is bundled in the IDE
 
VisualTeensy

luni said:
In case you are running Win10 you can give VisualTeensy a try. This will give you full control over the build process https://github.com/luni64/VisualTeensy
Click to expand...

Luni, what you have done with VisualTeensy looks very good. My development environment is Ubuntu Linux. I tried to run VisualTeensy using Wine(the Windows emulator for Linux) but sadly it failed. Not everything runs under Wine.

After having tried all the popular programming IDEs I settled on Geany. It has just the right combination of features for my programming style. There is a lot of hype about Vscode but I thought Geany was better(different strokes for different folks) For building my project I use the following Bash file. It is called from within Geany. You will see that I use arduino-builder rather than native make commands, a much simpler and more compatible approach. But then I am tied to the library priorities dictated by arduino-builder. This is an acceptable compromise for me, as long as I understand them, hence this post.

Build:
Code: 
#!/bin/bash
#teensy builder

BASE_PATH=$1
TARGET_PATH=$2
TARGET_NAME=$3
VERBOSE=$4

p0="-dump-prefs"
p1="-compile"
p2="-logger=human"
p3="-hardware $BASE_PATH/../installation/hardware"
p4="-hardware $HOME/.arduino15/packages"
p5="-tools $BASE_PATH/../installation/tools-builder"
p6="-tools $BASE_PATH/../installation/hardware/tools/avr"
p7="-tools $HOME/.arduino15/packages"
p8="-built-in-libraries $BASE_PATH/../installation/libraries"
p9="-libraries $BASE_PATH/../libraries"
p10="-libraries $BASE_PATH/libraries"
p11="-libraries $TARGET_PATH/libraries"
p12="-fqbn=teensy:avr:teensy40:usb=serial,speed=600,opt=o2std,keys=en-us"
p13="-ide-version=10812"
p14="-build-path $TARGET_PATH/build"
p15="-warnings=default"
p16="-build-cache $TARGET_PATH/cache"
p17=""
p18="$TARGET_PATH/$TARGET_NAME" 

echo "$VERBOSE Compile started, watch this space..."

if [ "$VERBOSE" == "-verbose" ];
then
  p17="-verbose"
  $HOME/Arduino/installation/arduino-builder $p0 $p2 $p3 $p4 $p5 $p6 $p7 $p8 $p9 $p10 $p11 $p12 $p13 $p14 $p15 $p16 $p17 $p18
fi

$HOME/Arduino/installation/arduino-builder $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8 $p9 $p10 $p11 $p12 $p13 $p14 $p15 $p16 $p17 $p18
echo "Compile completed"
echo "Project: $1"
echo "Folder: $2"
echo "Target: $3"

Upload 1 - teensy_post_compile:
Code: 
#!/bin/bash
# upload Teensy

BASE_PATH=$1
TARGET_PATH=$2
TARGET_NAME=$3

echo "Teensy upload"
echo $BASE_PATH
echo $TARGET_PATH
echo $TARGET_NAME


#TARGET_PATH="projects/flowsen-02"
#TARGET_NAME="flowsen-02.ino"

p1="-file=$TARGET_NAME"
p2="-path=$TARGET_PATH/build"
p3="-tools=$BASE_PATH/../installation/hardware/tools"
p4="-board=TEENSY40"
p5="-reboot"
p6="-port=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7.4/1-7.4.1"
p7="-portlabel=/dev/bus/usb/001/054 Bootloader"
p8="-portprotocol=Teensy"

$HOME/Arduino/installation/hardware/tools/teensy_post_compile $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8

echo "Upload completed"
exit

Upload 2 - teensy_loader_cli
Code: 
#!/bin/bash
#teensy loader

BASE_PATH=$1
TARGET_PATH=$2
TARGET_NAME=$3

echo "Teensy upload"
echo $BASE_PATH
echo $TARGET_PATH
echo $TARGET_NAME

echo "Press the Teensy load button"
$HOME/Arduino/teensy_loader_cli/teensy_loader_cli -mmcu=imxrt1062 -w  $TARGET_PATH/build/$TARGET_NAME.hex
echo "Load completed"
 
Corrected teensy_post_compile scripy

PDOS said:
Upload 1 - teensy_post_compile:
Code: 
#!/bin/bash
...
p6="-port=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7.4/1-7.4.1"
...
Click to expand...

This is an update to my teensy_loader Bash script, above.
I had hard coded the USB port address as follows
p6="-port=/sys/devices/pci0000:00/0000:00:14.0/usb1/1-7/1-7.4/1-7.4.1"
which is a bad practice since it changes if I plug into a different USB port.

Now I detect the USB port where the Teensy device is plugged in.
Below is the corrected script
Code: 
#!/bin/bash
# upload Teensy

BASE_PATH=$1
TARGET_PATH=$2
TARGET_NAME=$3

echo "Teensy upload"
echo $BASE_PATH
echo $TARGET_PATH
echo $TARGET_NAME

suffix="/manufacturer"
string=`ack -s -l "Teensyduino" "/sys/devices/pci0000:00"`
TARGET_PORT=${string%"$suffix"}
echo $TARGET_PORT


#TARGET_PATH="projects/flowsen-02"
#TARGET_NAME="flowsen-02.ino"

p1="-file=$TARGET_NAME"
p2="-path=$TARGET_PATH/build"
p3="-tools=$BASE_PATH/../installation/hardware/tools"
p4="-board=TEENSY40"
p5="-reboot"
p6="-port=$TARGET_PORT"
p7="-portlabel=/dev/bus/usb/001/054 Bootloader"
p8="-portprotocol=Teensy"

$HOME/Arduino/installation/hardware/tools/teensy_post_compile $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8

echo "Upload completed"
 
Status
Not open for further replies.
Back
Top