Accuracy, Precision and Tolerance

A vernier caliper measuring a cut metal piece on a workbench; an illustrative AI-generated photograph.

OpenAI image generation / AI-generated

Closeness to a reference, consistency of results, and acceptable limits—explained through cutting 5 cm metal pieces.

What are accuracy and precision?

Accuracy concerns closeness to a true or relevant reference value. Precision concerns agreement among repeated results under specified conditions.

For manufacturing, agreement with a design target is often described as cutting accuracy. For measurement, accuracy concerns agreement between a measured value and the true value of the quantity being measured. The design target and the actual length of a finished piece are separate values.

Formal measurement accuracy is a qualitative concept. Numerical statements are made about error, bias, uncertainty or specified performance limits. VIM: measurement accuracy.

How do the terms differ?

TermMeaningExample for a 50 mm target
AccuracyCloseness to the relevant true or reference valueHow closely the produced length agrees with the target, or a measurement agrees with the piece's actual length
PrecisionAgreement among repeated results under stated conditionsRepeated readings of the same stable piece cluster closely; readings across different pieces also include production variation
ToleranceLimits of acceptable variation specified by the design50 ± 0.10 mm permits 49.90–50.10 mm
RepeatabilityPrecision under specified repeatability conditionsConsistency during a short run with the same setup
TruenessAgreement of the long-run measurement average with a reference valueWhether repeated measurements are centred on a reference
BiasAn estimate of systematic measurement error; by extension, a systematic process offset when discussing productionA measuring instrument reads high, or a cutting process tends to cut long

Measurement precision is commonly described using measures of spread, such as standard deviation. A smaller spread indicates better precision under the stated conditions. VIM: measurement precision.

Repeatability conditions normally keep the procedure, operator, measuring system, location and relevant operating conditions the same over a short period. Consistency across days, operators or systems requires the changed conditions to be stated. VIM: repeatability.

Is a correct average enough?

A set of widely scattered results can have an average close to the reference. This does not establish that individual results are accurate.

For repeated measurements, trueness concerns the limiting average relative to a reference; precision concerns scatter. A small sample's average is an estimate and can be close to the reference by chance. VIM: trueness.

Likewise, a tightly grouped set can be systematically displaced. Bias and spread describe different features. VIM: measurement bias.

What is intended precision?

Intended or design precision describes required or claimed consistency. Observed or achieved precision describes consistency supported by observations.

These are useful descriptive phrases. They are not two additional formal metrology quantities.

StatementMore explicit interpretation
“Each piece must be within 0.10 mm of 50 mm.”A dimensional tolerance
“The standard deviation of cut lengths must not exceed 0.02 mm under the specified test.”A process precision requirement
“The machine adjusts in 0.01 mm increments.”A command or positioning increment
“The sampled readings have a standard deviation of 0.015 mm.”An observed sample spread, subject to sampling and measurement limitations

A specification states what is required. A manufacturer's claim states expected performance under stated conditions. Observations provide evidence of what was achieved.

A tolerance alone does not predict the distribution of results. A precision requirement alone does not establish agreement with the target.

What does 1 mm mean in centimetres?

1 cm = 10 mm, so 1 mm = 0.1 cm.

A tolerance of ±1 mm around 5 cm therefore gives limits of 4.9 cm and 5.1 cm.

The values 4.999 cm and 5.001 cm are only ±0.001 cm = ±0.01 mm = ±10 µm from the target.

“Precision of 1 mm” is incomplete. The specification must identify whether 1 mm refers to a standard deviation, a range, a maximum deviation, a positioning increment or another quantity. A claim about “most pieces” also needs a defined proportion and supporting distributional evidence.

What do two very similar readings show?

Suppose two pieces give readings of 5.010001 cm and 5.010006 cm.

The following arithmetic treats the stated readings as numerical inputs. The extra digits are retained to explain their relationships; they do not establish the capabilities of a real measuring instrument.

QuantityCentimetresMillimetres
Design target5.00000050.000000
Piece A reading5.01000150.100010
Piece B reading5.01000650.100060
Mean reading5.010003550.100035
Mean minus target+0.0100035+0.100035
Difference between readings0.0000050.000050
Sample standard deviation, using n − 1≈0.00000354≈0.0000354

For two readings, the sample standard deviation is their absolute difference divided by √2.

The readings differ by 0.000050 mm = 0.050 µm = 50 nm. Their mean is 0.100035 mm above the target.

The supported description is: these two readings agree closely with one another and both lie above the target.

What cannot be inferred from two pieces?

Two readings provide a very weak estimate of continuing process variation. The sample standard deviation has only one degree of freedom.

They do not establish long-term repeatability, process stability, a systematic cutting bias or the proportion of future pieces that will meet a tolerance. One measurement per piece also cannot separate measurement variation from actual variation between pieces.

The positive mean offset may reflect a cutting offset, a measurement offset, other effects or a combination. Repeated sampling and a characterised measuring system are needed to investigate the cause.

A 50 nm difference over approximately 50 mm is demanding to substantiate for cut metal. Suitable resolution alone is insufficient; the measurement method and uncertainty must support the comparison.

Are the pieces acceptable?

Acceptance depends on the tolerance and the decision rule.

Against ±1 mm limits, both indicated values lie inside the permitted interval. Against ±0.01 mm limits, both lie outside it.

Against ±0.10 mm limits, the upper limit is 50.100000 mm. The readings exceed it by only 10 nm and 60 nm respectively. Their numerical positions are clear, but a defensible acceptance decision requires measurement uncertainty and an agreed rule for decisions near the boundary.

What follows?

Specify the target, acceptable limits and required consistency separately. Report observed spread with its conditions and sample size. Keep statements about the cutting process distinct from statements about the measuring system.

Where can I read more?

Measurement, resolution and uncertainty explains what a reading tells us about a piece. Significant figures and rounding explains how many digits to retain.

Ai disclosure: written with the help of AI (ChatGPT). You are encouraged to point out errors and omissions.

Updated: 2026 Oct 30