Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Re: Averaging
    From: Jim Thompson
    Date: 2004 Oct 22, 07:46 -0300

    So, is this a fair summary?
    Sextant sights are subject to a variety of errors, leading to imprecision
    and inaccuracy. One way to deal with random observational error is to
    average a set of several sights, and then reduce and plot the average time
    and altitude.
    Always apply basic principles to a run of sights:
    1. Use only sights taken within a few minutes, a minute or less between
    2. Use either the raw sextant observations, or reduce each observation and
    use the reduced set. With modern programmable caculators, handheld computers
    and laptops, it is very easy for navigators to reduce every sight
    individually, and then use the reduced sights, rather than average the raw
    observations. This elimates variation owing to the way the corrections
    themselves might vary, but it still does not make the run of sights linear.
    3. Inspect the set of sights for consistency and discard obvious outliers,
    or sights that fail to increase and decrease in time or altitude reasonably
    smoothly. It helps to plot the sights on a graph of altitude by time, or
    build a table and use arithmetic to calculate the change in time and
    altitude between sights.
    4. Average time by adding up the minutes and seconds and dividing by the
    number of sights, then adding that average to the whole hour.
    5. Average altitude by adding up the minutes and seconds and dividing by the
    number of sights, then adding that average to the whole degrees.
    6. Work out the DR position of the average time based on vessel speed and
    course, and use the latitude and longitude of that position to reduce the
    average (if not done prior to averaging) and plot the resulting azimuth and
    Simple arithmetic averaging method improves precision in most cases, but
    beware some pitfalls:
    1. The body is very likely to be changing altitude in a nonlinear fashion.
    Altitude changes parabolically at the meridian, and most linearly when the
    body is on the prime vertical (due east or west). Over a period of a few
    minutes this is unlikely to be an issue for most common sights, but can lead
    to significant error in some situations. For runs of sights of typcial
    bodies taken over five minutes and under, arithmetic averages are subject to
    a systematic error of about 0.1 to 0.5 nautical miles, owing to nonlinearity
    in body motion. (IS THIS CORRECT?:) In the very worst case scenario, 2
    minutes before and after a body goes through the zenith, the error
    introduced by averaging might be up to 30 nautical miles, but this is a very
    rare situation. The errors are worse in some situations, and averaging
    should not be used in certain circumstances. The first two of the following
    rules were suggested by Alex on the Nav-L list:
    1a. Use arithmetic averaging if the body is lower than 60?.
    1b. If the body is over 60?, then use averaging if the azimuth is > 20? from
    the meridian.
    1c. Use arithmetic averaging only if all sights are obtained within about
    about 5 minutes.
    2. High vessel speed can also lead to non-linearity in the data. This is
    unlikely to be a problem over a few minutes at typical sailboat cruising
    speeds, but might need to be considered in very fast ships.
    Jim Thompson
    Outgoing mail scanned by Norton Antivirus
    > -----Original Message-----
    > From: Navigation Mailing List
    > [mailto:NAVIGATION-L@LISTSERV.WEBKAHUNA.COM]On Behalf Of Alexandre
    > Eremenko
    > Sent: Thursday, October 21, 2004 8:46 PM
    > Subject: Re: Averaging
    > I would like to re-iterate one point for those NOT
    > mathematically inclined. (Mathematicians frequently use
    > the words "and" and "or" in a precise technical sense,
    > which may be different from their daily use by other people).
    > So I restate the rule in the form of an algorithm
    > (or in the style of tax form instructions).
    > 1. Look at the altitude. If less than 60d you can average safely.
    > 2. If the altitude is more than 60 degrees look at the asymuth.
    > If the body is 20 degrees away from S and 20 degrees away from N,
    > you can average safely.
    > So the only "prohibited zone" consists of the
    > two sectors with vertices
    > at zenith, 30 degrees long and 40 degrees wide, whose
    > bisector is the meridian.
    > Alex.

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site