Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 11 of 11

Thread: map() function improvements

  1. #1
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,364

    map() function improvements

    Arduino's map() function has always left quite a bit to be desired. They've had a long standing bug report about the issues, and of course numerous times people have discussed using floating point.

    Well, I decided it's time to put a better map() function into Teensyduino. At least I hope it's better....

    For a little background, here's a simple test program:

    Code:
    void setup()
    {
    	while (!Serial) ;
    	Serial.println("map function test");
    	long in_min = 0;
    	long in_max = 7;
    	long out_min = 0;
    	long out_max = 3;
    	for (int i=in_min-5; i <= in_max+5; i++) {
    		Serial.print(i);
    		Serial.print(" --> ");
    		Serial.println(map(i, in_min, in_max, out_min, out_max));
    	}
    }
    
    void loop()
    {
    }
    This is supposed to map 0-7 onto 0-3. But here's what you actually get:

    map function test
    -5 --> -2
    -4 --> -1
    -3 --> -1
    -2 --> 0
    -1 --> 0
    0 --> 0
    1 --> 0
    2 --> 0
    3 --> 1
    4 --> 1
    5 --> 2
    6 --> 2
    7 --> 3

    8 --> 3
    9 --> 3
    10 --> 4
    11 --> 4
    12 --> 5
    So the first big problem is integer truncation is handled poorly.

    The other big problem is not support at all for floats. If the variable you're trying to map to a new range is a float, it gets converted to an integer and then put through this poor integer math.

    After much experimenting with C++ function overloading, I finally ended up with a C++ template solution using C++11 type trairs and SFINAE. The template syntax looks horribly complex.

    https://github.com/PaulStoffregen/co...58d2e438233f12

    With this approach we can have st42's proposed improvement to the integer algorithm, which is:

    Code:
            if ((in_max - in_min) > (out_max - out_min)) {
                    return (x - in_min) * (out_max - out_min+1) / (in_max - in_min+1) + out_min;
            } else {
                    return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
            }
    and when the input number is a float or double, the original math gets computed as 32 or 64 bit floating point.

    The integer case becomes:

    map function test
    -5 --> -2
    -4 --> -2
    -3 --> -1
    -2 --> -1
    -1 --> 0
    0 --> 0
    1 --> 0
    2 --> 1
    3 --> 1
    4 --> 2
    5 --> 2
    6 --> 3
    7 --> 3

    8 --> 4
    9 --> 4
    10 --> 5
    11 --> 5
    12 --> 6
    Now float can finally be used for map too:

    map function test
    -5 --> -2.14
    -4 --> -1.71
    -3 --> -1.29
    -2 --> -0.86
    -1 --> -0.43
    0 --> 0.00
    1 --> 0.43
    2 --> 0.86
    3 --> 1.29
    4 --> 1.71
    5 --> 2.14
    6 --> 2.57
    7 --> 3.00

    8 --> 3.43
    9 --> 3.86
    10 --> 4.29
    11 --> 4.71
    12 --> 5.14
    This change is scheduled to become part of Teensyduino 1.37.... unless you talk me out of it.

    Or maybe there's some way the integer round off could be handled even better? Is st42's way really the best? It seems better than all the other alternatives, but it does round towards zero and gives more values at zero when the target range spans negative to positive.

  2. #2
    Junior Member
    Join Date
    Sep 2015
    Posts
    9
    Great work!

  3. #3
    Senior Member
    Join Date
    Jul 2014
    Posts
    1,878
    Quote Originally Posted by PaulStoffregen View Post
    Arduino's map() function has always left quite a bit to be desired. They've had a long standing bug report about the issues, and of course numerous times people have discussed using floating point.

    Well, I decided it's time to put a better map() function into Teensyduino. At least I hope it's better....
    My 2 cents:
    I would call this not a bug, as done by st42 in Arduino forum.
    The method he proposed is constructed for the application where he wanted equal distribution of positive values

    a counter example extends to negative input values

    Code:
    long mmap0(long x, long mn1,long mx1, long mn2, long mx2)
    {
      return mn2+(x-mn1)*(mx2-mn2)/(mx1-mn1);
    }
    long mmap1(long x, long mn1,long mx1, long mn2, long mx2)
    {
      return mn2+(x-mn1)*(mx2-mn2+1)/(mx1-mn1+1);
    }
    
    void setup() {
      // put your setup code here, to run once:
    
      while(!Serial);
      Serial.println("map function test");
      long in_min = -7;
      long in_max = 7;
      long out_min = 0;
      long out_max = 3;
      for (int ii=in_min; ii <= in_max; ii++) {
        Serial.print(ii);
        Serial.print(" --> ");
        Serial.print(mmap0(ii, in_min, in_max, out_min, out_max));
        Serial.print(" --> ");
        Serial.println(mmap1(ii, in_min, in_max, out_min, out_max));
      }
    }
    
    void loop() {
      // put your main code here, to run repeatedly:
    
    }
    generates
    Code:
    map function test
    -7 --> 0 --> 0
    -6 --> 0 --> 0
    -5 --> 0 --> 0
    -4 --> 0 --> 0
    -3 --> 0 --> 1
    -2 --> 1 --> 1
    -1 --> 1 --> 1
    0 --> 1 --> 1
    1 --> 1 --> 2
    2 --> 1 --> 2
    3 --> 2 --> 2
    4 --> 2 --> 2
    5 --> 2 --> 3
    6 --> 2 --> 3
    7 --> 3 --> 3
    IMO the modified map generates a more un-symmetric mapping (the '1's are more on the negative side)

    I'm not saying that st42's method is not solving his view of the world, but is not generally applicable.

    Additionally, his argumentation in terms of steps is not correct, what is relevant is the estimation of the slope.
    one can estimate the correct slope with any number of values. The problem is integer arithmetic.

    You asked for a better method.
    there can not be a universal solution to misbehaving integer arithmetic, it all depends on the context
    For example:
    I would prefer that both min and max are extreme values indicating saturation (e.g. ADC)
    mapping from 16 bit to 8 bit (for displaying) I would like to maintain that min and max are extreme values that only occur once.
    Both the old map and the new map will not be suitable, and I have to generate my own map function.

    So context matters and should be stated.

  4. #4
    Senior Member
    Join Date
    Jan 2013
    Posts
    843
    Quote Originally Posted by WMXZ View Post
    My 2 cents:
    Code:
    map function test
    -7 --> 0 --> 0
    -6 --> 0 --> 0
    -5 --> 0 --> 0
    -4 --> 0 --> 0
    -3 --> 0 --> 1
    -2 --> 1 --> 1
    -1 --> 1 --> 1
    0 --> 1 --> 1
    1 --> 1 --> 2
    2 --> 1 --> 2
    3 --> 2 --> 2
    4 --> 2 --> 2
    5 --> 2 --> 3
    6 --> 2 --> 3
    7 --> 3 --> 3
    IMO the modified map generates a more un-symmetric mapping (the '1's are more on the negative side)
    Are you serious? You are mapping a 15-value range onto a 4-value range. IMO, 99% of the people out there expect a fair distribution of mapped values. The original map only maps one value to '3'. The modified version maps 3 or 4 values to each target.

    I can't think of a use case where the original map function has the right behavior.

    IMO, the correct behavior of a map function that is useful for most cases is to match rounded floating point math as close as is reasonably possible with integer math.

    @Paul:
    decltype(x) is the wrong return type to use. If the out range is larger, you get truncation (e.g. mapping uint8_t -> [0, 1000]). You should probably use 'long'.

  5. #5
    Senior Member
    Join Date
    Jul 2014
    Posts
    1,878
    Quote Originally Posted by tni View Post
    Are you serious? You are mapping a 15-value range onto a 4-value range. IMO, 99% of the people out there expect a fair distribution of mapped values. The original map only maps one value to '3'. The modified version maps 3 or 4 values to each target.
    1) Yes, I' m serious, that I'm not convinced of THAT solution
    2) I'm the 1%
    3) ALL my applications require the lowest and highest value to be a catch-all of values below and above. How can you differentiate between good (within boundary) and bad (outside boundary) value?
    4) the example is made to be an example

  6. #6
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,364
    Quote Originally Posted by tni View Post
    IMO, the correct behavior of a map function that is useful for most cases is to match rounded floating point math as close as is reasonably possible with integer math.
    I agree, approximating the float result is very likely what most people want map() to actually do, despite the comment in Arduino's documentation about integer truncation.


    Quote Originally Posted by WMXZ View Post
    1) Yes, I' m serious, that I'm not convinced of THAT solution
    2) I'm the 1%
    I can appreciate many differing opinions. Indeed there are many subtle usage cases.

    But the topic at hand is what specifically to implement for a widely used Arduino API. My goal is to give everyone (or 99% of everyone) the most widely useful results they would intuitively expect.


    Quote Originally Posted by WMXZ View Post
    3) ALL my applications require the lowest and highest value to be a catch-all of values below and above. How can you differentiate between good (within boundary) and bad (outside boundary) value?
    Arduino's documentation specifically says map() "does not constrain values to within the range" and they recommend using constrain() either before or after map() if you need to limit the range.

    But again, these are generic Arduino functions meant to cater to the general population of Arduino users, most of whom are novices. The question at hand here is not "what does WMXZ need" for a particular application that's an unusual 1% case, but rather what is the best, most appropriate algorithm to use for the large population of Arduino & Teensy users who expect map to "re-map a number from one range to another" (Arduino's description).


    Quote Originally Posted by tni View Post
    decltype(x) is the wrong return type to use. If the out range is larger, you get truncation (e.g. mapping uint8_t -> [0, 1000]). You should probably use 'long'.
    Like this?

    https://github.com/PaulStoffregen/co...cc06b642b8962a

  7. #7
    Senior Member+ Theremingenieur's Avatar
    Join Date
    Feb 2014
    Location
    Colmar, France
    Posts
    1,907
    Quote Originally Posted by WMXZ View Post
    3) ALL my applications require the lowest and highest value to be a catch-all of values below and above. How can you differentiate between good (within boundary) and bad (outside boundary) value?
    In that specific case, a sigmoid or tanh approximation could be a better solution. But the map() function is (at least in my modest understanding) expected to do a simple linear mapping in the y = m*(x-c) style. You might compare the input value to the input bounds before applying the linear map() and do some separate "out-of-range" exception handling.

  8. #8
    Senior Member
    Join Date
    Jan 2013
    Posts
    843
    Quote Originally Posted by PaulStoffregen View Post
    decltype(x) is the wrong return type to use. If the out range is larger, you get truncation (e.g. mapping uint8_t -> [0, 1000]). You should probably use 'long'.
    Like this?

    https://github.com/PaulStoffregen/co...cc06b642b8962a
    Yes .

  9. #9
    Senior Member
    Join Date
    Jul 2014
    Posts
    1,878
    Quote Originally Posted by PaulStoffregen View Post
    The question at hand here is not "what does WMXZ need" for a particular application that's an unusual 1% case, but rather what is the best, most appropriate algorithm to use for the large population of Arduino & Teensy users who expect map to "re-map a number from one range to another" (Arduino's description).
    Sure, I do not need or have to like Arduino software and I'm capable of doing my own programming and others do not need/like my programs.

    But you asked
    Is st42's way really the best?
    and I simply tried to say that it depends on what you wanted to do and what you expect from the results.

    I will not go into the merit of avoiding float mathematics or other methods that are known from graphics for linear interpolation and I recall similar things have been discussed in this forum some time ago, but I restate that I'm not convinced that the st42 solution is the best. It is a nice trick like a lot other heuristic tricks.

  10. #10
    Member
    Join Date
    Oct 2015
    Location
    France
    Posts
    32
    Paul,

    Is there a possibility that your map() function improvement is not took into account when I'm using Xcode+embedXcode and Teensyduino 1.39 (latest version available for all the software) ? I have curious behaviour on map() when the var is reaching the limit (a uint8_t var) when incremented by an encoder. When I'm implementing manually your patch it does improve the global behaviour (which let me know that your map patch is not took into account), but does not solve the cycling of the value from around 250 to 0 even with the use of constraint() before or after the map().

    Thanks a lot
    Last edited by xtof; 09-19-2017 at 07:25 AM.

  11. #11
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    18,364
    I've never used Xcode+embedXcode.

    I can tell you the improved map() function is definitely in the 1.39 code. The old one is long gone. So unless there's somehow an old copy laying around in embedXcode, and it somehow using that old code instead of the latest (which all seems quite unlikely), I just don't understand how you could be seeing the old map() behavior.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •