# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: Consistency v Accuracy in celestial navigation
From: Chuck Taylor
Date: 2003 Dec 29, 21:32 -0800

```These comments are in response to Kieran Kelly's query
about consistency vs. accuracy vs. precision in the
context of celestial navigation.  Since I don't
possess a PhD in the subject, I don't claim to be a
professional statistician, but I do rub elbows with
professional statisticians on a daily basis while
testing statistical software.  And I did teach the
subject for a few years a couple of decades ago at the
Naval Postgraduate School in Monterey, California.

It seems to me that Kieran has a very good grasp of
the topic. One can certainly be consistently off in
one direction or the other in taking measurements.
Statisticians call this "bias".  In shooting an
altitude, for example, not holding the sextant quite
vertical would introduce a bias in that the measured
altitude would consistently be too high.

A "line of best fit" (which I would call a linear
regression line or a least-squares fit) is appropriate
when the errors in measurement are known to be random
(not biased) and symmetric.  (More precisely, the
assumption is that errors are distributed according to
a Gaussian or "Normal" distribution, but random and
symmetric is usually good enough.)  The classic
example of this situation in the context of celestial
navigation is taking sights from a pitching deck in
heavy weather. A given measurement is as likely to be
high as low.

There is an old story that the man with one watch
always knows what time it is, but the man with two
watches is never sure.  The story can be extended a
bit in the case of celestial sights.

With only a single sight, you have no choice but to
use what you have.

Two sights always plot in a straight line, so there is
little reason to choose one over the other.  Averaging
can help if you have reason to believe that any errors
are random.

Three sights can be very useful if they plot in a
straight line.  Then you at least can conclude that
they are consistent.  If they do not plot in a
straight line, you are stuck.  You can make a case
that two of them are consistent and the third one is
not, but you have no way of knowing *which* one is the
odd sight out.

With four sights, things start to get better.  More
than likely, at least three will plot more or less in
a straight line. If so, then you can probably write
off the fourth sight as being flawed.  Statisticians
would call this an "outlier".   Of course if all four
sights plot in a straight line, so much the better.

With five or more sights, the story is much the same
as with four.

The lesson to be learned here is that it is better to
plot your sights and visually evaluate their quality
before including them in any sort of averaging or line
of best fit.  Throw out any obvious outliers, *then*
do your averaging or least squares line.

All of the above is in the name of consistency. It
says nothing about accuracy, or lack of bias.

Besides a systematic error in measurement, other
factors can introduce bias.  One example is that
actual celestial (or terrestial) refraction may differ
(due to atmospheric conditions at your particular
location and time) from that which is shown in the
tables.  (Terrestial refraction affects dip of the
horizon.)  Kieran mentioned some possible sources of
bias in the text quoted below.

As Kieran pointed out, the best way to judge accuracy
is by comparing your computed results with known true
positions.  In real life, we often find out about
accuracy only in hindsight.  How close did we make our
landfall?  Another way is to compare positions
determined by celestial means to positions determined
by GPS or using bearings from known landmarks.  If a
particular navigator has a track record of accurate
sights (evaluated in hindsight), then one has reason
to expect that his future sights are likely to be
similarly accurate.

Kieran also mentioned precision, but didn't say much
about it.  Precision refers to how precise the
measurements are.  To how many decimal places do we
record our measurements?  For example, I have one
sextant whose scales can be read to 2 minutes of arc,
and another whose scales can be read to 0.2 minutes of
arc.  The latter allows you to take more precise
measurements than the former.

It may be, however, that a skilled navigator can take
sights that are more accurate but less precise with
the sextant with the 2-minute scale than a
less-skilled navigator could achieve using the sextant
with the 0.2-minute scale.  By "more accurate" I mean
he would end up with a computed position that is
closer to the true position.

In the case of sights taken from land, any errors may
or may not be random. In the absence of bias,
measurement errors generally are random.  Judgement is
required in deciding whether or not bias exists. If
bias is present, averaging or using a line of best fit
will not necessarily improve the accuracy of your
result.

By the way, I have found that computer spreadsheets
such as Excel can be very useful in plotting sights to
evaluate consistency.  You first need to convert your
times into hours and decimal fractions of an hour, and
your altitudes into degrees and decimal fractions of a
degree.  Then let the computer do the plotting.

I hope that this is the sort of discussion that Kieran
was looking for.

With best regards,

Chuck Taylor
47d 55.2 N
122d 11.2 W

--- Kieran Kelly  wrote:

> ... there may be
> some confusion between consistency, accuracy and
> precision in this ongoing discussion.
>
> When I applied the line of best fit technique to
> Gregory's analysis I was  trying to measure his
> consistency i.e. the deviation away from a line of
> best fit. This says nothing about accuracy. ...
>
> ... A
> line of best fit applied to
> a series of observations may be very consistent with
> a small standard
> deviation and may be also very inaccurate not taking
> account of systematic
> error such as shade error, incorrect calculation of
> index error, observer
> bias, or failure to account for temperature and
> pressure. Thus a series of
> observations may be consistent and also consistently
> wrong calculation of either GMT from a lunar or
> position from an astronomic
> fix.

__________________________________
Do you Yahoo!?
http://photos.yahoo.com/

```
Browse Files

Drop Files

### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

### Email Settings

 Posting Code:

### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)