# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: Averaging
From: Herbert Prinz
Date: 2004 Oct 21, 16:17 -0400

```Bill has brought to my attention that the table that I posted earlier may be misleading
in so far as it does not directly answer the question he asked, namely, "what is the
maximum error resulting from linear averaging of a given data set of altitude
observations?"

I had tabulated the difference between the arithmetic mean of two altitudes and the
actual altitude at the middle of the interval. In other words, the maximum error due to
linear interpolation. My purpose was to demonstrate the non linearity of the curve. To
find the answer to Bill's question, you have to apply the table in the data to your
actual sample. For instance, if you take three sights, each 2 minutes apart, you have 2
sights that each contribute an error of 14" and one that contributes 0". So the total
error is 66% of what you find in column 4 m.

For this purpose, the table was incomplete as it lacked the column for 2 minutes
separation; so I add it here (and remove the one for 5).

Alt             2m             4m             6m
60 deg         0'03"          0'14"          0'31"
75 deg         0'07"          0'30"          1'06"
80 deg         0'11"          0'45"          1'40"
85 deg         0'22"          1'30"          3'21"

Now, if you take five sights in 1 minute intervals, you add the entries in the
appropriate line from columns 2m and 4m  and multiply with 0.4 to find the error. For
seven sights you add the entries from all three columns and multiply with 2/7 or 0.3.

But remember, this is the MAXIMUM error. How bad it really is, say 30 deg of the
meridian, I addressed in my previous message.

Herbert Prinz

```
Browse Files

Drop Files

### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

### Email Settings

 Posting Code:

### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)