Round And Round

:date: 2024-03-12 10:51 :tags:

I'm no mathematician but I like to think I could comfortably score well on a third grade math test. And I would assume that my proficiency on grade school tests would transfer to the exact same applied problems as they arose in the real world. Today I was surprised to find this was not entirely true!

The topic is rounding numbers, something that is generally taught to eight year old kids. Let's belabor the topic with a couple of examples.

Imagine you buy a $77.00 thing and the sales tax in is 7.5%  —  how much will you pay in sales tax? Go ahead and trust me that the precise answer when calculating 77 times 0.075 is 5.775. But that's not how American money works and we must round. The grade school test question then is how many dollars and cents is shown on a sales receipt for this tax?

If you said $5.78, congratulations, you would ace a kid's math test!

Let's try another one. A different item is $211.00  —  now how much is the sales tax? You can again accept that the precise answer is 15.825 which again needs to be rounded. What is the dollars and cents value printed on the receipt for this tax?

If you said $15.83, congratulations, you would ace a kid's math test! Unfortunately, in some situations, you could be wrong!

If I have purchased a $211.00 item I can now very easily imagine that the receipt would show a 7.5% sales tax as $15.82.

First let me prove that there is a good reason to believe that the values would come out like this on a receipt. Receipts are created by software these days and most software is derived from other software that ultimately derives from something compiled by a C compiler. Consider this very simple C program.

[source,c] 

/* rounder.c - Demonstration of C's rounding behavior. */
#include <stdio.h>
int main() {
    float num1= 5.775, num2= 15.825;
    printf("%f: %.2f\n",num1,num1);
    printf("%f: %.2f\n",num2,num2);
    return 0;
}

I'm just assigning the two values I mentioned and printing them out at the correct number of decimals. If I compile and run that, look what we get. 

$ gcc -o round rounder.c && ./round
5.775000: 5.78
15.825000: 15.82

Awk is closely related to C and predictably follows. 

$ echo 5.775 15.825 | awk '{printf "%.2f %.2f\n",$1,$2}'
5.78 15.82

What about Bash? You can type this right into any terminal. 

$ printf "%.2f %.2f\n" 5.775 15.825
5.78 15.82

I'm guessing they probably all use a common library somehow.

What about Python? That's actually where I first noticed this and it has this behavior too. 

>>> round(5.775,2); round(15.825,2)
5.78
15.82

So this is clearly no quirk. For some reason all of these pretty serious computational systems seem to think that what you learned as a kid is not quite right. What I learned as a kid is that a value like 5.475 would be rounded up to 5.48, not down to 5.47. The rule I learned was if you are rounding and need to make a decision on a 5 you go up. In fact as a kid I learned an algorithm for programming purposes. Here it is in Applesoft BASIC as I remember it on my Apple ][+. 

PRINT INT(5.775*100+.5)/100

Basically, multiply the number by 100, add .5 (letting that carry), chop off all the decimal part with the INT, and divide by 100 again to put it back into the correct scale. I have spent 40 years thinking this is how rounding works.

Here's LibreOffice correctly imitating Excel, but it is somewhat interesting that at least in this example, it does not match the behavior of C and Python.

round-libreoffice.png

What is going on here? When I learned about rounding as a kid it would have been impossible for me and even my teachers to really learn much more about it. But today, of course there is an astonishingly rich Wikipedia page on the topic of rounding numbers.

It alerts me to the idea of "round to even". The official documentation for the Python round() function goes on to hint that is indeed the algorithm that is being used. It says.

...if two multiples are equally close, rounding is done toward the even choice (so, for example, both round(0.5) and round(-0.5) are 0, and round(1.5) is 2).

Note how this handles the weirdness of rounding negative numbers. I was taught that 24.5 rounded to 25 (rounding up) and -24.5 rounds to -24 (also rounding up in value). In Python (and similar) 24.5 rounds to 24 and -24.5 rounds to -24.

If you look over that Wikipedia page, you'll start to realize that rounding is much more nuanced than your third grade teacher made it seem. I'm realizing that it is like date/time issues where armies of very smart programmers  —  many of whom are mathematicians!  —  pull their hair out over the fussiest of details.

xkcd 2867

Best to just let them get on with it and if you see some receipt that doesn't seem to be rounded "correctly", well, it probably is!

UPDATE 2024-03-14 Or... Maybe not... At least not the "correctly" you were expecting. I just wrote a followup about this topic here.

Also from the 1980s... music to help you think about rounding.