Measuring instruments: range and precision

A measuring tape is ideal for a bench but useless for the thickness of a wire, and a stopwatch that shows hundredths of a second cannot beat your reaction time. A good measurement starts with an instrument whose range and precision suit the job. It then depends on reading the scale correctly, checking the zero, timing many oscillations, repeating readings and measuring many items to find one.

Range and precision

Two properties decide whether an instrument suits a measurement.

• The range runs from the smallest to the largest value the instrument can measure. A 30 cm rule measures lengths up to 30 cm; a 0–1 A ammeter measures currents up to 1 A.
• The resolution describes the smallest change the instrument can distinguish. For these school instruments, compare the smallest scale division or display step: a metre rule marked in millimetres has 1 mm divisions, while the stated digital calipers change in 0.01 mm steps. Fine resolution helps you distinguish small changes. Precision is how closely repeated measurements agree; accuracy is how close a measurement is to the true value. See the measurement vocabulary’s distinction.

Choose in this order:

1. Range: the range must cover the value you expect.
2. Scale or display step: one division must be small compared with the value, and with any difference you need to detect.
3. Practicality: the instrument must fit the object and be quick to use well.

A tape with 1 mm divisions can measure a 1.20 m bench as 1200 mm. A 1 mm division is tiny compared with this length. The same 1 mm is bigger than a wire 0.5 mm thick, so a rule cannot tell a 0.4 mm wire from a 0.6 mm one.

Instruments you should know

The ranges and resolutions are typical values. Read them from the instrument you actually use.

More digits do not make a reading correct. A precise instrument that is not zeroed gives readings that are precise but wrong.

Choosing a meter range

Many ammeters and voltmeters have more than one range. Suppose an analogue ammeter has a 0–1 A range and a 0–5 A range, each with 50 divisions across its scale.

• On 0–1 A, one division is 1 ÷ 50 = 0.02 A.
• On 0–5 A, one division is 5 ÷ 50 = 0.1 A.

To measure a current of about 0.35 A, both ranges cover it, but the 0–1 A range reads it to 0.02 A instead of 0.1 A, so choose 0–1 A. If you do not know the current, start on the largest range and switch down once you see the reading fits. A current larger than the range sends the pointer off the scale and can damage the meter.

Choose a voltmeter range the same way. For the p.d. across a 1.5 V cell, a 0–3 V range covers it; if the 0–3 V and 0–5 V scales have the same number of divisions, the 0–3 V divisions are smaller, so choose 0–3 V.

Read the scale correctly

A reading is only as good as the way it is taken. The end of a rule is often worn, so the rod below is lined up with a clear mark instead of the end.

The left end is at 2.0 cm and the right end is at 7.6 cm, so the rod is 7.6 - 2.0 = 5.6 cm long. The rule’s smallest division is 1 mm = 0.1 cm, so the length is recorded to 0.1 cm: 5.6 cm, or 56 mm. Reading only the right end would give 7.6 cm, too long by the 2.0 cm before the rod starts.

Look straight at each mark. Looking from one side makes the mark appear to move against the scale; this is parallax error.

In this cylinder, the labelled marks are 10 cm³ apart with five spaces between each pair, so one small division is 10 ÷ 5 = 2 cm³. Water curves up at the walls. Read the bottom of the curved surface, the meniscus, with your eye level with it: it is three small divisions above 30, so the volume is 30 + 3 × 2 = 36 cm³.

Read a thermometer the same way: keep the bulb in the liquid, stir, and read with your eye level with the end of the liquid thread.

Check the zero

Before you measure, check what the instrument reads when it measures nothing: calipers or micrometer closed, balance empty, spring balance hanging unloaded. If it does not read zero, it has a zero error, and every reading is shifted by that amount.

Reset it to zero if you can. Otherwise, correct each reading:

A zero error can be negative. A spring balance whose pointer rests at −0.2 N when unloaded reads 3.4 N with a load. The corrected weight is 3.4 - (-0.2) = 3.6 N: subtracting a negative zero reading adds to the reading.

Time many oscillations, not one

One complete oscillation of a pendulum is a swing from one side to the other and back. The time for one oscillation is the period.

A digital stopwatch may display 0.01 s steps, but hand-trigger variation is often much larger. Follow the recording convention in your practical task, and use repeats to assess the variation. The last displayed digit may change because of your reaction time.

A pendulum with a period of about 1.8 s shows the problem. A 0.2 s timing error is more than a tenth of a single period. So time many oscillations and divide.

Start the stopwatch as the bob passes the centre in a stated direction, counting that pass as “zero”. Count one complete oscillation each time it returns through the centre in the same direction. A pass in the opposite direction occurs only halfway through the cycle. The bob moves fastest at the centre, making the crossing easier to judge.

Repeat readings and take the mean

Readings of the same quantity differ a little each time: your timing varies, and a wire is never perfectly round. Take each reading at least three times and use the mean:

Before averaging, compare the readings. If a fourth reading were 34.6 s, it would be about one period short of the others, which suggests a miscounted oscillation. Check the count and repeat the measurement. Keep the original reading and record the investigation; do not exclude it merely because it differs. Exclude a reading only when the investigation identifies an error, such as an observed miscount.

For a wire, measure the diameter at several places along it and in two directions at right angles. Readings of 0.452 mm, 0.448 mm and 0.455 mm give a mean of 1.355 ÷ 3 = 0.452 mm.

Measure many to find one

When one item is too small compared with the instrument’s scale or display step, measure many comparable items together, then divide to estimate a typical item.

The same idea works on a balance: 50 identical paper clips with a mass of 23.5 g have a mass of 23.5 ÷ 50 = 0.47 g each.

Some quantities are worked out from other measurements. The volume of an irregular stone is found by displacement: water in a measuring cylinder reads 50 cm³, and 62 cm³ with the stone fully under water, so the stone’s volume is 62 - 50 = 12 cm³.

Common mistakes

• Choosing the instrument with the most digits. Check that the range covers the value first, then compare its scale or display step with the value and differences to detect.
• Expecting a more precise instrument to fix a zero error. Check the zero, then reset it or subtract the zero reading.
• Adding a positive zero error. The instrument already reads too high: corrected reading = reading − zero reading.
• Treating the stopwatch’s last digit as proof of accuracy. Time several complete oscillations and repeat; the hand-trigger variation can be larger than the display step.
• Forgetting to divide by the number of oscillations or items. That gives the total, not one.
• Reading the top of the meniscus, or looking from an angle. Read the bottom of the meniscus with your eye level with it.
• Reading only the far end of an object. Subtract the start reading from the end reading.
• Starting a meter on its smallest range. Start on the largest range, then switch down.
• Deleting an unusual reading without a justified, recorded reason. Investigate and repeat; keep the original result.