# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: Averaging
From: Jim Thompson
Date: 2004 Oct 22, 07:46 -0300

```So, is this a fair summary?

Sextant sights are subject to a variety of errors, leading to imprecision
and inaccuracy. One way to deal with random observational error is to
average a set of several sights, and then reduce and plot the average time
and altitude.

Always apply basic principles to a run of sights:

1. Use only sights taken within a few minutes, a minute or less between
sights.
2. Use either the raw sextant observations, or reduce each observation and
use the reduced set. With modern programmable caculators, handheld computers
and laptops, it is very easy for navigators to reduce every sight
individually, and then use the reduced sights, rather than average the raw
observations. This elimates variation owing to the way the corrections
themselves might vary, but it still does not make the run of sights linear.
3. Inspect the set of sights for consistency and discard obvious outliers,
or sights that fail to increase and decrease in time or altitude reasonably
smoothly. It helps to plot the sights on a graph of altitude by time, or
build a table and use arithmetic to calculate the change in time and
altitude between sights.
4. Average time by adding up the minutes and seconds and dividing by the
number of sights, then adding that average to the whole hour.
5. Average altitude by adding up the minutes and seconds and dividing by the
number of sights, then adding that average to the whole degrees.
6. Work out the DR position of the average time based on vessel speed and
course, and use the latitude and longitude of that position to reduce the
average (if not done prior to averaging) and plot the resulting azimuth and
intercept.

Simple arithmetic averaging method improves precision in most cases, but
beware some pitfalls:

1. The body is very likely to be changing altitude in a nonlinear fashion.
Altitude changes parabolically at the meridian, and most linearly when the
body is on the prime vertical (due east or west). Over a period of a few
minutes this is unlikely to be an issue for most common sights, but can lead
to significant error in some situations. For runs of sights of typcial
bodies taken over five minutes and under, arithmetic averages are subject to
a systematic error of about 0.1 to 0.5 nautical miles, owing to nonlinearity
in body motion. (IS THIS CORRECT?:) In the very worst case scenario, 2
minutes before and after a body goes through the zenith, the error
introduced by averaging might be up to 30 nautical miles, but this is a very
rare situation. The errors are worse in some situations, and averaging
should not be used in certain circumstances. The first two of the following
rules were suggested by Alex on the Nav-L list:

1a. Use arithmetic averaging if the body is lower than 60?.
1b. If the body is over 60?, then use averaging if the azimuth is > 20? from
the meridian.
1c. Use arithmetic averaging only if all sights are obtained within about

2. High vessel speed can also lead to non-linearity in the data. This is
unlikely to be a problem over a few minutes at typical sailboat cruising
speeds, but might need to be considered in very fast ships.

Jim Thompson
jim2@jimthompson.net
www.jimthompson.net
Outgoing mail scanned by Norton Antivirus
-----------------------------------------

> -----Original Message-----
> Eremenko
> Sent: Thursday, October 21, 2004 8:46 PM
> Subject: Re: Averaging
>
>
> I would like to re-iterate one point for those NOT
> mathematically inclined. (Mathematicians frequently use
> the words "and" and "or" in a precise technical sense,
> which may be different from their daily use by other people).
>
> So I restate the rule in the form of an algorithm
> (or in the style of tax form instructions).
>
> 1. Look at the altitude. If less than 60d you can average safely.
> 2. If the altitude is more than 60 degrees look at the asymuth.
> If the body is 20 degrees away from S and 20 degrees away from N,
> you can average safely.
>
> So the only "prohibited zone" consists of the
> two sectors with vertices
> at zenith, 30 degrees long and 40 degrees wide, whose
> bisector is the meridian.
>
> Alex.
>
>

```
Browse Files

Drop Files

### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

### Email Settings

 Posting Code:

### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)