Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Re: Averaging
    From: Jim Thompson
    Date: 2004 Oct 23, 08:03 -0300

    I just spent the last couple of hours reading the entire set of messages in
    this thread, at the Nav-L archives.  Thanks to that, and to the message Alex
    just sent, I have revised my summary.  How does this look now?
    Jim Thompson
    Outgoing mail scanned by Norton Antivirus
    Averaging Sights
    Sextant sights are subject to a variety of errors that lead to imprecision
    and inaccuracy. This is a serious concern for a navigator who is relying on
    celestial navigation to find his or her position at sea, making comfortable
    sleep difficult. One way to deal with random observational error is to
    average a set of several sights, and then to plot the average time and
    altitude. See also Bowditch article 1609, and the Nav-L thread "averaging"
    in October 2004.
    Always apply basic principles to a run of sights:
    1. Use 3-5 sights taken within a few minutes, a minute or less between
    2. Use raw sextant observations, before applying corrections.
    3. Inspect the set of sights for consistency and discard obvious outliers,
    or sights that fail to increase and decrease in time or altitude reasonably
    smoothly. It helps to plot the sights on a graph of altitude over time; or
    build a table, use arithmetic to calculate the change in time and altitude
    between sights, and then inspect the table for outliers.
    And then, to average a run of acceptable sights:
    4. Average time by adding up the minutes and seconds and dividing by the
    number of sights, then adding that average to the whole hour.
    5. Average altitude by adding up the minutes and seconds and dividing by the
    number of sights, then adding that average to the whole degrees.
    6. Work out the DR position of the average time based on vessel speed and
    course, and use the latitude and longitude of that position to reduce the
    average and plot the resulting azimuth and intercept.
    Simple arithmetic averaging improves precision in most cases by taking out
    random error, but beware some pitfalls:
    1. Averaging does not take out bias errors caused by problems in the sextant
    or observer.
    2. A celestial body's altitude changes in a non-linear fashion. But,
    fortunately for navigators, altitude non-linearity in most of the celestial
    window is smaller than our ability to detect it with an observational device
    like a hand sextant, which in the hands of most of us allows a precision of
    about than 0.5' of arc. The change in altitude over short time durations of
    5-10 minutes is so nearly linear that for our purposes we can assume it is
    linear. The important exception to this is when the body passes through the
    observer's meridian at high altitudes, over about 60?-75?. The change in
    altitude is significantly non-linear in that case. For runs of sights of
    typical bodies taken up to five minutes, nonlinearity introduces a
    systematic error of only 0.1 to 1.0 nautical miles.  The error is in the
    lower half of that range under 60? of altitude, and greater than 20? from
    the meridian. In the very worst case scenario, 2 minutes before and after a
    body goes through the zenith, the error introduced by averaging is up to 30
    nautical miles, but this is a very rare situation.
    In the usual sight-taking altitude-azimuth window, navigators can average
    their sights, comfortable that they are taking out random error without
    introducing a signficant error caused by non-linearity in the way the
    altitude changes during sight-taking. Averaging should not be used in
    certain circumstances. I obtained these rules from a long thread about
    averaging on the Nav-L list in October 2004. If these rules are followed,
    then the systematic error owing to non-linearity in the change of altitude
    over 5 minutes will be less than 1' of arc, or 1 nautical mile (0.5' arc and
    0.5 NM if the navigator uses the 60? and 20? limits):
    1. Use arithmetic averaging if the body is lower than 60?-75?.
    2. If the body is over 60?-75?, then use averaging only if the azimuth is
    >10?-20? from the meridian, and do not use simple arithmetic averaging if
    the body is closer to the meridian at those high altitudes.
    3. Use arithmetic averaging only if all sights are obtained within about
    about 5-10 minutes.
    Using Computers Instead of Averaging
    With modern programmable caculators, handheld computers and laptops, it is
    very easy for navigators to reduce every sight individually and then plot
    all the reduced sights, rather than average a run of raw observations and
    plot just the average sight. The navigator can then plot the 3-5 acceptable
    (consistent) reduced sights for each body and graphically find the best
    single intercept between them. Following this method, the navigator does not
    have to take sights on the same body before moving to another body, but can
    shoot bodies as the opportunity arrives, coming back to earlier bodies to
    take additional sights if the horizon is still good.
    And as Gary LaPook wrote on Nav-L during the averaging thread, "As long as
    we are talking about the St. Hilaire method (computing an azimuth and
    intercept from an assumed position) we should remember that it was developed
    as an easy method of laying down the "Sumner Line," now called an LOP,
    requiring only one computation. The original Sumner method required
    computing two time sights, twice as much work. With programmable calculators
    it is now just as easy to do the two computations and lay down the LOP
    without measuring an azimuth or intercept or using an assumed position. You
    simply choose two longitudes, one east and one west of your DR, and the
    calculator calculates the latitude where the LOP crosses those longitudes.
    You prick those latitudes on the chart and draw a straight line between
    The disadvantage of the computer methods is the risk of "blunder", of
    entering a wrong data element into the computer, so that the reduced sight
    ends up being wrong, even though the raw observation might have been
    excellent. The risk of such blunders increases with the number of sights
    reduced, especially for a fatigued navigator in a sailboat with a small
    But there are many similar ways to think outside the box when so much
    computational power is available to modern celestial navigators.

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site