Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Re: Consistency v Accuracy in celestial navigation
    From: Chuck Taylor
    Date: 2003 Dec 29, 21:32 -0800

    These comments are in response to Kieran Kelly's query
    about consistency vs. accuracy vs. precision in the
    context of celestial navigation.  Since I don't
    possess a PhD in the subject, I don't claim to be a
    professional statistician, but I do rub elbows with
    professional statisticians on a daily basis while
    testing statistical software.  And I did teach the
    subject for a few years a couple of decades ago at the
    Naval Postgraduate School in Monterey, California.
    It seems to me that Kieran has a very good grasp of
    the topic. One can certainly be consistently off in
    one direction or the other in taking measurements.
    Statisticians call this "bias".  In shooting an
    altitude, for example, not holding the sextant quite
    vertical would introduce a bias in that the measured
    altitude would consistently be too high.
    A "line of best fit" (which I would call a linear
    regression line or a least-squares fit) is appropriate
    when the errors in measurement are known to be random
    (not biased) and symmetric.  (More precisely, the
    assumption is that errors are distributed according to
    a Gaussian or "Normal" distribution, but random and
    symmetric is usually good enough.)  The classic
    example of this situation in the context of celestial
    navigation is taking sights from a pitching deck in
    heavy weather. A given measurement is as likely to be
    high as low.
    There is an old story that the man with one watch
    always knows what time it is, but the man with two
    watches is never sure.  The story can be extended a
    bit in the case of celestial sights.
    With only a single sight, you have no choice but to
    use what you have.
    Two sights always plot in a straight line, so there is
    little reason to choose one over the other.  Averaging
    can help if you have reason to believe that any errors
    are random.
    Three sights can be very useful if they plot in a
    straight line.  Then you at least can conclude that
    they are consistent.  If they do not plot in a
    straight line, you are stuck.  You can make a case
    that two of them are consistent and the third one is
    not, but you have no way of knowing *which* one is the
    odd sight out.
    With four sights, things start to get better.  More
    than likely, at least three will plot more or less in
    a straight line. If so, then you can probably write
    off the fourth sight as being flawed.  Statisticians
    would call this an "outlier".   Of course if all four
    sights plot in a straight line, so much the better.
    With five or more sights, the story is much the same
    as with four.
    The lesson to be learned here is that it is better to
    plot your sights and visually evaluate their quality
    before including them in any sort of averaging or line
    of best fit.  Throw out any obvious outliers, *then*
    do your averaging or least squares line.
    All of the above is in the name of consistency. It
    says nothing about accuracy, or lack of bias.
    Besides a systematic error in measurement, other
    factors can introduce bias.  One example is that
    actual celestial (or terrestial) refraction may differ
    (due to atmospheric conditions at your particular
    location and time) from that which is shown in the
    tables.  (Terrestial refraction affects dip of the
    horizon.)  Kieran mentioned some possible sources of
    bias in the text quoted below.
    As Kieran pointed out, the best way to judge accuracy
    is by comparing your computed results with known true
    positions.  In real life, we often find out about
    accuracy only in hindsight.  How close did we make our
    landfall?  Another way is to compare positions
    determined by celestial means to positions determined
    by GPS or using bearings from known landmarks.  If a
    particular navigator has a track record of accurate
    sights (evaluated in hindsight), then one has reason
    to expect that his future sights are likely to be
    similarly accurate.
    Kieran also mentioned precision, but didn't say much
    about it.  Precision refers to how precise the
    measurements are.  To how many decimal places do we
    record our measurements?  For example, I have one
    sextant whose scales can be read to 2 minutes of arc,
    and another whose scales can be read to 0.2 minutes of
    arc.  The latter allows you to take more precise
    measurements than the former.
    It may be, however, that a skilled navigator can take
    sights that are more accurate but less precise with
    the sextant with the 2-minute scale than a
    less-skilled navigator could achieve using the sextant
    with the 0.2-minute scale.  By "more accurate" I mean
    he would end up with a computed position that is
    closer to the true position.
    In the case of sights taken from land, any errors may
    or may not be random. In the absence of bias,
    measurement errors generally are random.  Judgement is
    required in deciding whether or not bias exists. If
    bias is present, averaging or using a line of best fit
    will not necessarily improve the accuracy of your
    By the way, I have found that computer spreadsheets
    such as Excel can be very useful in plotting sights to
    evaluate consistency.  You first need to convert your
    times into hours and decimal fractions of an hour, and
    your altitudes into degrees and decimal fractions of a
    degree.  Then let the computer do the plotting.
    I hope that this is the sort of discussion that Kieran
    was looking for.
    With best regards,
    Chuck Taylor
     47d 55.2 N
    122d 11.2 W
    --- Kieran Kelly  wrote:
    > ... there may be
    > some confusion between consistency, accuracy and
    > precision in this ongoing discussion.
    > When I applied the line of best fit technique to
    > Gregory's analysis I was  trying to measure his
    > consistency i.e. the deviation away from a line of
    > best fit. This says nothing about accuracy. ...
    > ... A
    > line of best fit applied to
    > a series of observations may be very consistent with
    > a small standard
    > deviation and may be also very inaccurate not taking
    > account of systematic
    > error such as shade error, incorrect calculation of
    > index error, observer
    > bias, or failure to account for temperature and
    > pressure. Thus a series of
    > observations may be consistent and also consistently
    > wrong leading to a
    > wrong calculation of either GMT from a lunar or
    > position from an astronomic
    > fix.
    Do you Yahoo!?
    New Yahoo! Photos - easier uploading and sharing.

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site