NavList:

   

Reply

From a theoretical viewpoint, it is wrong to average the observations
before reducing them.

Alexandre is basically right that the motion of the observer alone on a
small sailing vessel during a relatively short interval, say 5 min, has
little impact on the fix. But his analysis addresses the wrong problem.
It is the motion of the observed object that really matters. If this
motion is noticeably non-linear - and for an altitude observation it
almost always is - averaging may lead to errors that exceed the expected
random errors of measurement (see one exception, below). The motion of
the observer aggravates the error further. In other words: The costs may
be higher than the benefit. A position error of  1.5 nm due to naive
averaging of altitudes over 5 minutes would not be atypical.

If one takes more observations than are required to obtain a fix, the
latter becomes overdetermined. The only rigorous solution to this
problem is to minimize the sum of the squares of the individual unknown
observation errors. The correct procedure is given in Chauvenet's
"Manual" (in the appendix), in the Nautical Almanac (pp. 277-283), in
the HMNAO publication "AstroNavPC and Compact Data". Discussions of
various algorithms can be found in several articles in the Journal of
Navigation.

In the few special cases where the observandum is a linear function of
time, averaging is appropriate. The main candidate for this is the time
sight, when the sun (or other object) is in the prime vertical. The
other application is the lunar distance, provided the geometry of moon
and second object is favourable. Theoretically, meridian transits also
fall into this category when, for a brief time span, the altitude of the
object changes less than what can be nominally resolved on the
instrument. This is only relevant for land based latitude determination,
since the observations have to be confined to the precomputed time
interval.

For the said three applications, which were important at one time, Borda
and others constructed repetition circles, a type of instrument that
implements averaging in hardware.  Different kinds were used
successfully by French navigators and, more so, by land surveyors. Their
characteristic advantage is that they accumulate the measured angle, so
one does not have to read the instrument after each individual
observation. Only the time of each observation is recorded, and finally
one reads the sum of the angles.

Yet an other mechanical averaging device can be found on bubble
sextants. As these were used for general altitude observations, they
seem to be in conflict with the stated rule. But the comparatively large
random error (which may actually be periodic when used on an aircraft)
justifies the integration (= averaging) of the observed altitude during
a brief interval, typically 2 minutes. The shorter interval makes for a
better linear behaviour of the change in altitude during the
observation.

Since both methods, the time sight as well as the lunar distance have
fallen into total oblivion, manual averaging is a thing of the past.

Before I address the idea of rejecting spurious data, let me clearify
the terminology.

What Alexandre refers to as "human error", I call a "blunder". This is
not to be confused with "personal error", which is systematic and can be
detected in the long run. A blunder, on the other hand, is an isolated
occurrence of a gross error that can be detected within one individual
data set on account of its size. Most common sources are procedural
error, the introduction of a wrong number into any of the elements of
the computation other than the observandum, a wrong reading of the
latter, or improper use of the instrument. (E.g. picking a wrong date in
the almanac, wrong sign, reading the wrong number of degrees in
conjunction with a Vernier or micrometer reading, or the micrometer
screw did not snap in properly after moving the index arm.)

Of course, we would like to eliminate blunders. The problem is to
identify them correctly. Not every spurious data point is a blunder. It
is statistically not sound to remove out-liers as a matter of principle.
For the typically small data sets that appear in celestial navigation,
what are the sound, applicable criteria that tell us which observation
to discard and which to keep? I do not think they exist. Therefore I do
not feel good about removing out-liers if they cannot clearly be traced
back to procedural errors.

By the way, the repeating circle leaves no possibility whatsoever to
remove out-liers due to bad measurement, but inherently eliminates
blunders stemming from misreading of the individual observation.

Much that has been written on the subject of error elimination is geared
towards the needs of the astronomer or land surveyor and not applicable
to celestial navigation. To give but one drastic example of how
different the problems are: Finding latitude by meridian transit of a
star. The navigator has a few minutes to do this, the astronomer a full
year. The navigator seeks to find the extrem value during one transit
(averaging would be pervers!), whereas the astronomer can apply the
whole statistical apparatus to many such observations (one per day) of
upper and (half a year later) lower transits of the same circumpolar
star.

Many navigators develop a "feeling" for the quality of an observation
and decide right then and there, at the moment they take it, whether to
keep it. But if one is not sure about an observation, why not include it
in the set, reduce it with the other ones and then see how the minimized
error residual for that particular observation fits in the overall
picture? What is the point of spending the extra effort of plotting it
beforehand?

Herbert Prinz


Alexandre Eremenko wrote:

> Many manuals advise to take several sights
> in short sequence and then to average the result.
> and to reduce the average as one sight.
> (Chauvenet recommends at most 6, Russian manuals 3-5).
> The purpose is to increase accuracy,
> and, probably more importantly, to reject the sights
> with an evident "human error".
>
> [...]

   

Reply