The Error Of Significant Figures
:date: 2026-10-03 20:06
Decades ago when I first took a chemistry class the very first topic changed my life — and not in a good way. The topic was significant figures. At first it didn't make sense to me but as time went on, it still never made sense. That was a red flag. When I took basic chemistry in college — same syllabus, same misgivings. When a friend was taking basic chemistry while I was working in a machine shop with my degree in Industrial Engineering complete, my misgivings were becoming convictions. There was something deeply wrong, or at least impractically facile, about how significant figures were portrayed in beginner chemistry classes.
I feel like the Introductory Chemistry Significant Figures (let's call it ICSF) topic is well meaning but deeply flawed. ICSF does nothing to help one understand chemistry per se. A very strange and important question is: why is it not taught before biology or physics? Why has this been awkwardly bolted on to chemistry? Perhaps it is imagined that it could help with cooking recipes in your kitchen and I suppose the idea is that will at some point help with cooking recipes in a chem lab producing a concomitant chemistry understanding. But the problem is that it is just a muddled approximation that will lead to epistemological confusion when engineering problems become serious.
What is the problem with this? Here is XKCD showing an example of one issue.
It has been difficult for me to articulate exactly what the problem is because I never properly learned what these ICSF heuristics were. (To demonstrate I am not just imagining this, here is an example of ICSF learning materials and another.) I feel like I got through chemistry by trying to guess what a capricious insane person with a gun wanted me to say. (And then I had to spend the rest of the course being regarded as a "bad student" by what to me was a capricious insane person.)
I've wanted to write this post for decades, but I also didn't want to think too hard about this topic. Today I realized that our robot friends can do the unpleasant donkey work and maybe I can get some closure for this educational trauma.
The Problem
What exactly is the ICSF method anyway? It is intended to restrain unwarranted digit exuberance when representing and using numbers. For example, if I ask what's the speed limit or how many grams of nuts does this recipe call for, it is not optimal for the answer to be "45.00000000000000". More subtle problems arise if I tell you I have a cube that holds about a gallon and you tell me the sides must be 6.135792439661958 inches. Clearly that is not a good way to communicate or think. The ICSF is a well intentioned attempt to manage this.
If my chemistry teacher had pointed this problem out and told us to use good judgment and avoid excessive precision, well, that would have been fine. Instead ICSF was introduced and presented with the exact credibility and authority as, say, the Pythagorean Theorem.
ICSF Precisely Defined
Here we come to the part of this exploration where I do not have enough shits to give to dig into the muddled technique I was taught. This is where our robot friends are perfect for summarizing the normal thing that chemistry students are normally taught. I'm putting its summary here for completeness, but it is probably better for you philosophically to just skim past this without thinking too hard.
Counting significant figures
* Nonzero digits always count.
* Zeros between nonzero digits count (1005 has 4).
* Leading zeros never count (0.0045 has 2).
* Trailing zeros count only if there's a decimal point (100. has 3,
1.50 has 3). In 100 they're placeholders, which is the ambiguous
case. Scientific notation resolves it: 1.00E2.
* Exact numbers (counts, defined conversions like 100 cm/m) have
infinitely many.
Arithmetic
* Multiply/divide: the result gets as many sig figs as the input with the fewest.
* Add/subtract: the result is rounded to the least precise decimal
position among the inputs.
* Round only at the end, carrying extra guard digits in the middle.
* Logs: the digits after the decimal point in pH (the mantissa) equal
the sig figs in [H⁺]. So pH 4.74 means [H⁺] has 2 sig figs.Is this the "correct" heuristic? Maybe. When talking about textual number conversion algorithms, I personally like to communicate in a proper language designed for such things. Our robot friend helpfully translated that for me into this more serious expression of what exactly ICSF is. Again just skip right past this unless you're taking a very keen interest, but this is an extraordinarily explicit description of what is taught to beginning chemistry students.
from decimal import Decimal, ROUND_HALF_UP
import re
class Val:
"""A number plus the position of its last significant digit."""
def __init__(self, d, pos): self.d, self.pos = d, pos
def n(self): return self.d.adjusted() - self.pos + 1 # sig figs
def _prod(self, d, other):
n = min(self.n(), other.n())
return Val(d, d.adjusted() - n + 1)
def __mul__(a, b): return a._prod(a.d * b.d, b)
def __truediv__(a, b): return a._prod(a.d / b.d, b)
def __add__(a, b): return Val(a.d + b.d, max(a.pos, b.pos))
def __sub__(a, b): return Val(a.d - b.d, max(a.pos, b.pos))
def __str__(self):
if self.pos < -50: return str(self.d) # exact
q = self.d.quantize(Decimal(1).scaleb(self.pos), ROUND_HALF_UP)
if q.adjusted() > self.d.adjusted(): # 9.99 -> 10, not 10.0
q = self.d.quantize(Decimal(1).scaleb(self.pos + 1), ROUND_HALF_UP)
return str(q)
def S(s):
"""Parse a number as written, e.g. S('0.0450'), S('100.'), S('1.5E3')."""
d = Decimal(s)
mant = re.split('[eE]', s)[0].lstrip('+-')
digits = mant.replace('.', '').lstrip('0')
if '.' not in mant: digits = digits.rstrip('0') # trailing-zero convention
return Val(d, d.adjusted() - max(len(digits), 1) + 1)
def E(s): return Val(Decimal(s), -999) # exact number
# Some tests...
print(S('12.11') + S('18.0') + S('1.013')) # 31.1 (limited by 18.0)
print(S('4.2') * S('3.14159')) # 13 (2 s.f.)
print(E('2') * S('3.14')) # 6.28The point is that this is the algorithm I was expected to run in a virtual computer in my brain when I was a kid. I suspect this is mostly because it is something that involves "math" and can be easily graded. It is essentially a MacGuffin side quest of chemistry education.
The Problem With The Solution
ICSF is essentially a logarithmic proxy for relative uncertainty, and
it is a questionable one. To start with, the ROUND_HALF_UP assumption
in the program is just that, an assumption. (I have written before how
computer rounding is much harder than
you might think.)
There are also some different variants of ICSF. For example, is "100" three significant digits? Or one? Some ICSF systems clumsily insist on "100." to establish the full three. What if I want two?
Here are some reasonable problems my robot friend could enumerate about ICSF. It also was able to generate impressive plots based on actual extemporaneous simulation experiments to visually show the differences between ICSF (tan) and ground truth reality (blue).
* The 1.0 vs 9.9 cliff, now in logarithms - [H⁺] = 1.0×10⁻³ and [H⁺] =
9.9×10⁻³ both have 2 sig figs, so both pH values get 2 decimals: pH
3.00 and pH 2.00. But 1.0 is ±5%, which is ±0.02 in pH, so "3.00"
overclaims by 4×. And 9.9 is ±0.5%, which is ±0.002 in pH, so "2.00"
underclaims by 2×. Same rule, wildly different honesty.
* The rules violate the distributive law - (12.5 − 12.0) × 3.000: the
subtraction gives 0.5 (1 sf), and times 3.000 gives 1.5, which
rounds to 2. Distribute it instead: 12.5×3.000 − 12.0×3.000 = 37.5 −
36.0 = 1.5. Same arithmetic, two different answers. The true value
is about 1.50 ± 0.02, so the first answer is off by a factor of 25
in its implied tolerance.
* Round trips don't return home (the xkcd effect) - 5 mph → 2 m/s
(2.235, rounded to 1 sf) → 4 mph (4.47, rounded). One unit
conversion each way and you've lost 20% of the value, even though
both conversion factors are exact. The sig-fig interval for "5 mph"
is 4.5–5.5 mph, but the interval for "2 m/s" is 1.5–2.5 m/s, which
is 3.4–5.6 mph. The rule changes the implied tolerance every time
the unit changes.
* It can't recognize that x − x = 0 or x/x = 1 - A sample is weighed
once as 0.250 g. The mass fraction of Mg in pure Mg is 0.250/0.250 =
"1.00 (±0.5%)", when it is exactly 1. Any quantity appearing in both
numerator and denominator (or subtracted from itself) is perfectly
correlated. Errors cancel exactly, and ICSF has no way to say so.
* Adding many terms understates uncertainty - Add 100 readings from a
balance that resolves 0.1 g. ICSF says the total is good to 0.1 g
(implied ±0.05). In reality each reading carries about 0.03 g of
standard uncertainty. The sum's uncertainty is √100 × 0.03 ≈ 0.3 g,
six times the implied tolerance, and in the worst case ±5 g.
* Averaging many terms overstates uncertainty - Ten titrations read
12.3, 12.4, 12.3, 12.2, 12.3, 12.4, 12.3, 12.3, 12.4, 12.3 mL. The
mean is 12.32, the standard deviation is 0.063, and the standard
error of the mean is 0.02. ICSF says to report 12.3 and throw away
the extra digit, which discards real information. (This works only
because the readings actually scatter; identical readings can't be
averaged past the resolution.) Examples 5 and 6 together show the
rule is wrong in both directions depending on the operation.
* Digits are not accuracy - A Class A 100 mL volumetric flask is
certified to ±0.08 mL. Writing "100.00 mL" implies ±0.005 mL, which
is a 16× overclaim. Conversely, the rule would give a 100 mL
graduated cylinder reading of "100. mL" a clean bill of health it
hasn't earned. ICSF reads the number of digits you wrote down, not
the instrument, calibration, or certificate.
* "100" means anything from ±0.5 to ±50 - Is "100 mL" 1, 2, or 3 sf?
Textbooks disagree. At 1 sf the implied range is 50–150 mL. Multiply
by a density of 1.234 g/mL and you get either 1×10² g or 123 g, a
23% spread caused entirely by an unwritten convention.
* Half-up rounding builds in a bias - Instrument readings 0.5, 1.5,
2.5, 3.5, 4.5 sum to 12.5. Rounded half-up, that's 1+2+3+4+5 = 15
(+20%). Round-half-even gives 0+2+2+4+4 = 12 (−4%), and unbiased on
average. Many ICSF courses teach half-up as the rule, and the
instructor may or may not know that other conventions exist.
* Zero has no significant figures - By rule 3, leading zeros never
count, so a balance reading "0.000 g" after taring has zero sig
figs. Yet it is a precise statement: the mass is within ±0.0005 g of
zero. Multiply it by anything and the rules give you an answer with
no defined precision. Meanwhile 0.0010 g and 0.001000 g are
described as having 2 and 4 sig figs, but nothing in the rules
explains why "less than a milligram" should be treated as a
different kind of number from "nothing."
Better Solutions
The best solution to ICSF specifically would be to write this on the board:
"Calculations, especially computer aided, can impart a false sense of accuracy. Use good judgment to avoid false accuracy problems."
Problem solved. At a high school chemistry level this is fine. But what of those who go on to become chemists or some profession which, apparently, takes accuracy even more seriously?
If you're seriously serious, like NASA or NIST serious, you probably want to know about the ISO's Joint Committee For Guides In Metrology. They have a subgroup, the Working Group on the Statement of Uncertainties, which publish JCGM 100. This document's rigor and seriousness clearly highlight the deficiencies of ICSF methods. This specification has very detailed rationale for how to report values (how many digits do you show) and also for how to propagate values (how calculations are most accurately performed). A related standard is called "GUM (ISO/IEC Guide 98-3)"; GUM is "Guide to Uncertainties in Measurement" which certainly seems relevant.
Internalizing this standard is the way scientists at metrology labs must think about these problems, but there are more practical engineering approaches. A way these problems are competently solved is with an error budget. With this technique a table is made of uncertainty inputs along with their sensitivity coefficients and its share of the variance. This allows one to pay closer attention to the measurements that have the biggest impact on the desired outcomes.
Once the uncertainty has been managed it often needs to be communicated (the "reporting" part). While ICSF does this with clumsy digit counting, proper engineering documentation should use ASME Y14.5 - Dimensioning and Tolerancing
It was when I first saw geometric dimensioning and tolerancing used in engineering praxis that I became certain the ICSF heuristic I had been taught was badly flawed.
But things can get even worse. What if your uncertainty is nonlinear or even stochastic? For example what if I need to build a chain out of several batches of links whose variable size follow different complex distributions? Simply adding the links' lengths may not suffice. Some kind of Monte Carlo simulations may be needed to get an idea of the error of a full assembly.
Why This Bugs Me
So that's my rant about Introductory Chemistry Significant Figures (ICSF). Why do I care so much?
I formally studied a STEM discipline that takes accuracy more seriously than chemistry seems to. My specific degree, industrial engineering, is the sub-specialty that explicitly focuses on such matters. As I became aware of more sophisticated ways to think about uncertainty, I could reevaluate my first weeks of chemistry class and see how deficient it was. I've got a whole different rant about the deficiencies of typical frequentist and Bayesian statistical methods generally, but at least with that we're waiting for another Einstein level genius to fix it. With ICSF, it is simple to demonstrate a better way.
A huge part of my early career in manufacturing was all about taking accuracies seriously. As the factory computer nerd, I was often the ground truth. I learned to use a laser interferometer so that I could provide some of the highest accuracy our customers had access to. The more seriously I studied accuracy, the more ICSF became a joke.
Later in my career as a serious computer nerd I worked with some of the world's most brilliant molecular biophysicists and biochemists at one of the top biotech universities. The painful irony was that my chemistry potential had been deeply stunted by my second week of high school chemistry. In those two weeks, I went from being interested in a career in plastics engineering to insisting on a career that had nothing whatsoever to do with chemistry. (Oh well.) All because of this topic which chemistry writ large would have done well to leave to the experts.
I now see it as some kind of counterproductive hazing. If it was simply a matter of not being able to run that Python script in my brain perhaps I would have considered the game fair. But even as a 15 year old kid I could smell the epistemological problems with ICSF. This was badly exacerbated by the teacher, let's call him Fundy McBlowhard, who was a serious and frequent violator of The Law. I'm thankful that my robot friends allowed me to explore the reality of the situation better without the need to seriously study a questionable thing that I probably should not have ever studied.