A CNC miller hands over a part that a cutter has already cut. A drawing that says ±10 μm or ±1 μm measures how far the holes, lengths, or contours on that part deviate from the drawing. The resolution, encoder counts, and controller decimals in a machine catalog answer a different question: can the command get one finer step?
This article answers one question: what do the 10 μm and 1 μm on a drawing actually measure? The answer: three quantities (positioning accuracy A, repeatability R, and workpiece dimension) answer different questions; 10 μm maps to the compensated two-axis path, 1 μm maps to the compensated single axis, and a workpiece holding the same numbers still needs temperature, cutting force, tooling, and measurement added on top.
1. Three quantities: positioning accuracy, repeatability, and workpiece dimension
| Quantity | What it asks | What 1 μm means here |
|---|---|---|
| Positioning accuracy A | On a single axis, the system bias plus scatter of the actual position vs. the command | The band around the commanded point after axis compensation |
| Repeatability R | Returning to the same target the same way, how scattered are the landings | Scatter around the old point, excluding bias from the drawing origin |
| Workpiece dimension | After chips stop, the fixture opens, and temperature returns to drawing conditions | That ±10 μm or ±1 μm on the drawing |
ISO 230-2 defines A as the range combining bidirectional system bias and positioning scatter; R is the width of the landing scatter[3]. With small R, a machine can park steadily at a wrong position — printing R on a quotation does not answer the hole diameter.
ISO 230-2:2014 measures the relative motion between the tool-carrying structure and the workpiece-carrying structure, valid for linear and rotary axes[3]. So the A a laser interferometer walks along X is that one axis’s positioning; circular interpolation and the ballbar belong to the ISO 230-4 family of tests. Both happen before the cutter has finished this part. The workpiece’s physical accuracy adds one more stage on top of those two reports: cutting force, tool wear, clamping, heat, and the CMM’s own uncertainty.
Figure 1 | The accuracy layers from command point to workpiece. Click for full size.
2. Two published machine tests
A production 3-axis miller (Sarvas et al., 2024): positioning per ISO 230-2, circles per ISO 230-4 ballbar. Uncompensated, the X-axis positioning error was 20.5–33.1 μm; Table 5 at three Z heights (−460/−405/−350 mm) and three feeds (500/3000/8000 mm/min) gives nine A values spanning 23.5–33.1 μm — A moves with feed and Z[1].
The compensation test separates the layers: at Z = −350 mm, A = 29.5 μm with compensation off, 1.4 μm with the correct-height table on; with the wrong-height table, A = 7.7 μm. At Z = −460 mm the same pattern: 21.4 μm off, 1.4 μm with the right table, 7.6 μm with the wrong one; repeatability R = 0.8–2.5 μm. In a warm-up test, A with compensation on landed at 1.3–1.7 μm[1].
The ballbar did not follow down to 1 μm: the ballbar synthesizes the two axes’ motion into a circle, measuring the path instead of a single axis. At the same Z, 3000 mm/min feed, three thermal states, positioning tolerance was 18.7/20.7/22.9 μm and circularity 11.3/10.7/10.6 μm with compensation off; with it on, positioning tolerance 12.6/9.8/9.7 μm and circularity 8.5/7.3/6.6 μm — the two-axis positioning tolerance stayed at about 10–13 μm[1].
A precision micro-milling centre (Mijušković et al., 2013): the manufacturer’s axis tolerance was 3 μm. Before compensation, bidirectional A was: X (420 mm) 7.76 μm, Y (350 mm) 5.99 μm, Z (200 mm) 2.24 μm — X and Y beyond the manufacturer’s tolerance; all three axes’ R under 0.7 μm. After correcting the pitch-error compensation tables, A became 1.06/0.85/0.65 μm, with R at the same level[2]. The 1 μm here is still an axis, with strokes of only a few hundred millimeters, on a tool-and-die-class micro-milling centre — the interferometer measured the axis, not a milled die core.
3. Heat: the dominant error term
The geometric compensation table matches the screw or optical-scale error at the moment of measurement; once the machine runs, heat pushes the structure further. Li et al.’s 2021 review of thermal-error modeling cites Bryan 1990: thermal error is about 40%–70% of a machine’s total machining error; and Ramesh et al. 2000: thermal error reaches about 60%–70% of the total[4]. Mayr et al.’s 2012 CIRP keynote states that up to about 75% of a workpiece’s overall geometric error can be caused by thermal effects[5]. These percentages are review ranges, used to say heat is the dominant term — not that every part is exactly 70%.
Convert 1 μm into temperature with thermal expansion: steel’s linear expansion coefficient in engineering handbooks is about (10.8–12.5)×10⁻⁶ /°C[6]. Taking 12×10⁻⁶ /°C: 100 mm of steel rising 1 °C elongates 1.2 μm; 300 mm of steel, the same 1 °C, elongates 3.6 μm. A 300 mm scale warms by under 0.3 °C and the elongation is already 1 μm. After the compensation table pulls the axis A down to 1.4 μm, room temperature, the spindle, and the coolant can still push the part out of that band.
4. How to write the two tolerance levels on a drawing
10 μm class: hole positions, contours, two-axis features. It maps to the compensated ballbar on Sarvas’s miller (positioning tolerance about 9.7–12.6 μm, circularity about 6.6–8.5 μm)[1]. Acceptance needs a compensated circle test or an equivalent workpiece measurement — not just a single-axis A sheet. This layer still excludes cut depth and tool deflection: the ballbar is a low-load circle, and once the cutter takes material, force pushes the spindle, holder, and workpiece a little further, so the physical dimension is wider than the air circle. A 10 μm drawing is a “the motion is already at 10 μm, and cutting still needs its own margin” tolerance — not setting the controller resolution to 10 μm.
1 μm class: short-stroke, mostly single-axis, temperature-controlled, low-cutting-load features. It maps to two published axis results: Sarvas’s compensated single-axis A of 1.3–1.7 μm (1.4 μm with the correct-height table) and the micro-milling centre’s compensated A of 0.65–1.06 μm with R under 0.7 μm[1][2]. Acceptance numbers come from the workpiece, with the measurement uncertainty stated; a single-axis A at 1 μm is one necessary condition. For a whole part to hold 1 μm, the drawing environment, machine environment, and measurement environment must share one temperature regime — a 100 mm steel feature moves 1.2 μm per 1 °C.
Repeatability: write it into the stability of the program and fixture, not into the dimensional tolerance as a substitute for A.
The previous article was about the cutter itself: nose, edge, and material set how thin a chip can be. The 10 μm and 1 μm in this article are where the machine delivers the cutter — however sharp the tool, with the compensation table at the wrong height the circle stays at the 10 μm level.
Back to the opening question: the 10 μm and 1 μm on a drawing are two workpiece-accuracy levels with test evidence — 10 μm for the compensated two-axis path, 1 μm for the compensated single axis; a workpiece holding the same numbers still needs its own measurement. Scope: the numbers here come from one 3-axis miller and one micro-milling centre (specific strokes, temperatures, conditions) — not a catalog specification that applies to every CNC. Unresolved limits: neither paper provides a comparison measurement of “finished part, batch dimension spread equals A”; the thermal-error 40%–70% and up-to-75% figures come from reviews, not from a decomposition of these two machines.