NavList:

A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: On potential error introduced by rounded values
From: George Huxtable
Date: 2005 Jan 9, 16:45 +0000

```In the past, when Peter Fogg and I have discussed Bennett's "The Complete
On-Board Celestial Navigator", we have disagreed strongly: but not this
time. I agree with the general thrust of his argument, with his numerical
model, and his conclusions, though there might be a bit of quibbling about
detail.

Because, in Bennett's book, all the tabulations are only to the nearest
whole minute, many "rounding" operations, to the nearest minute, are called
for. Each such rounding introduces an error, at the most ? 0.5', but any
value within that range is equally probable. Statistically, this is a
"square" or "flat" error distribution, just 1' wide, differing markedly in
shape from the familiar Gaussian, or "bell curve" error distribution.

If there are two such roundings, they need to be combined by a "folding"
operation, which results in a triangular error distribution, with a
base-width of 2', and at the 50% level being 1' wide. Already, this is
looking a bit more like the shape of a bell curve.

Each successive rounding operation that's folded in brings the resulting
error distribution closer to the shape of a bell curve, and widens it
slightly. But the resulting width doesn't increase in proportion to the
number of roundings, but to the square root of that number. Even so, the
maximum conceivable error, if everything conspires in the same direction,
is still ? 0.5 x the number of roundings. However, the more such roundings
there are, the less likely it will be that they will all work, in the same
direction, to their maximum extent: this is Peter's argument, and a valid
one.

Peter suggests- "There are 5 or 6 values (6 for stars, 5 for other bodies)
entered during the sight reduction process used in George Bennett's book",
and I won't argue with that assessment. If we take 6 such roundings, then
in theory the maximum amount they could possibly (but most unlikely)
contribute to an error in position is ?3 miles, though in practice the
error is unlikely to exceed about ?1.5 miles, as Peter's numerical model
shows.

The question then arises: what does a prudent navigator take as the error
introduced by the roundings in this calculation process, to combine with
his estimated errors of observation? How much skirting distance should he
allow around a charted but unmarked rock in mid-ocean, relying only on
astro observations to avoid it? How narrow a passage between two such
hazards would he tackle? A careful family-man might take the view that if
there's a possibility (even a tiny one) of the calculation being 3 miles
out either way, that's what he should allow for. A gambler might assess
that if there's no more than (say) a 1 in 50 chance of the calculation
error exceeding 1.5 miles, he will accept the risk and plot his course in
that basis. After all, we're all risk-takers to some extent, or we wouldn't
be out there on small boats in big waters.

Of course, no navigator makes such a numerical hazard-assessment, or has
the detailed information on which to base it: but still, those are the
estimations that should be going through his mind, perhaps unconsciously.

Peter offers two numerical models. One draws a parallel with the numbers of
boys and girls in a family. I suggest that this is a very over-simplified
approach to the problem, presuming that errors would always be the same in
amount with just the sign being random, and his numerical results are
therefore not very relevant. However, Peter's second model, using random
numbers, is to my mind a good analogy with the real-life error situation.
The results are meaningful, and the simulation could usefully be taken
further.

Peter's conclusion, that " rounding to whole numbers in a series does not
lead to a great chance of the rounded amounts adding up to significant
error." is somewhat woolly, but I can see what he's getting at.

He states- "The result of this sight reduction is an intercept that is
expressed to a whole minute of arc, leading to a fix consisting of latitude
and longitude expressed as whole minutes of arc." Which is fair enough, as
long as the user appreciates that a resulting intercept, given to an
arc-minute, may be in error; that the true answer could very likely be the
adjacent minute either way, and much less likely, yet another minute away
than that.

He ends- "I suspect there is also some wooly thinking involved in the
assumption that a fix expressed to the nearest tenth of a minute of arc is
better for the purpose of position finding from a small boat ..."

Agreed. The Nav-l list frequently concerns itself with a quest for
perfection, in observing to the nearest 0.1'. However, such ambitions
crumble when faced with the reality of a small boat on an unquiet sea with
an indefinite horizon. I accept that the standard of precision for
intercepts, achieved in the Bennett tables (generally speaking, to an
arc-minute or two) is perfectly appropriate for small craft, when viewed in
the light of the limited precision achievable in such observations. The
important proviso is that these limitations should be understood, and
accepted, by the user.

================

Finally, after all that accord, and if only to tweak Peter Fogg's tail a
little, I can't resist pointing out that earlier criticisms of the Bennett
tables revolved around the major errors in azimuth, of many degrees, that
can occur when using Bennett's azimuth tables for directions anywhere near
to due East or West: not the errors of a minute or two in intercept that
are being considered at present.

George.

================================================================
contact George Huxtable by email at george@huxtable.u-net.com, by phone at
01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy
Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK.
================================================================

```
Browse Files

Drop Files

Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

Email Settings

 Posting Code:

Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)