# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: accuracy of sights and averaging**

**From:**Frank Reed

**Date:**2010 Dec 5, 23:30 -0800

Alan, you wrote:

"Turns out that the intercept, difference between Hc and Ho were quite large, much larger than when individual shots were reduced.

Assuming that I haven't blown the "math", I haven't noticed any glaring computational errors, where might the problem lie? "

That's pretty much the only option here, Alan, as far as I can see. Just go over it again, and I bet you will find a simple arithmetic error. The average of the intercepts should give nearly the same result as the single intercept from the average of all the sights. Note that this should be done only with a set of sights taken in rapid succession over a very short period of time.

Various people have been advocating plotting a line with the known slope, the rate of increase of the object's altitude, from some assumed position and then looking at the differences between the plotted altitudes and that line. This is actually a very reasonable procedure since it is nothing more than a simple, fast method for calculating Hc through the run of sights. The idea is that, when the sights are relatively close together in time and azimuth, the Hc increases at a nearly linear, steady rate. So you can just draw a line on a piece of graph paper displaying that linearly-increasing rate, plot the values of Ho at known times on the same graph, and then directly "read off" Ho-Hc as the various distances between the Ho points and the Hc line. The values you get are nearly identical to the intercept values. You can calculate this sloping line either by using a unique equation (not complicated, but you have to dig it up and remember it every time) OR, NEARLY AS EASY, and generally more accurate, you can just calculate the Hc at some convenient time near the beginning of your set of sights and then calculate Hc again for a time near the end of the set of sights. You plot those two points and draw a straight line through them with a straight-edge. The distance between the line and the Ho points are the intercepts. Very simple and very effective. This is a fast way of clearing a moderately large number of sights acquired over a short period of time when calculating devices are unavailable.

Using a system like this to detect and remove "outliers" (bad sights) is a tougher call, but definitely viable, as Peter Fogg has said, if the outliers are really far out of line. You can't just pare away any and all sights that are inconsistent with a nice line. You need to use a careful and rigorously applied system for removing bad sights, and in many cases, it's better to keep all the sights since the appearance of an "outlier" may be an illusion, a random accident. That's why any such removal system has to be decided on in advance and applied according to the rule without exception.

By the way, for the mathematically-inclined: for those who would like to experiment with rules for deleting outliers, you can easily create sets of simulated sights to test out various systems. But you will need a method of "seeding" your simulated data with outliers. This can be done by "mixing" random numbers taken from two normal (Gaussian) distributions with significantly different standard deviations. For example, with some probability f, say, 80% of the time, you take your random numbers from a distribution that has a mean of 0.0 and a standard deviation of perhaps 0.7 minutes of arc. This represents normal sights with normal random error. Then with probability 1-f (whatever is left over), you take your random numbers from a distribution that has a mean of 0.0 and a standard deviation of maybe 2.5 minutes of arc. This represents "dodgy" sights. In an Excel spreadsheet, you can use these formulas: in A1-A10 (for 10 sights), enter =NORMINV(RAND(),0,0.7). In B1-B10, enter =NORMINV(RAND(),0,2.5). In C1-C10, enter =IF(RAND()>0.75,B1,A1). These simulated random errors can then be added on to pre-calculated altitudes, for example. You can then experiment with various rules for throwing out outliers.

-FER

----------------------------------------------------------------

NavList message boards and member settings: www.fer3.com/NavList

Members may optionally receive posts by email.

To cancel email delivery, send a message to NoMail[at]fer3.com

----------------------------------------------------------------