NavList:

The article on "The Accuracy of Astronomical Observations at Sea" posted by Andrés talks a lot about the unexpectedly large number of outliers in celestial navigation data compared to the so-called "normal" distribution. This is an important feature of celestial navigation sights. It's "normal" for them to be abnormal, and there's an ugly technical name for this statistical behavior (which I have written about before).

Gaussian or "normal" distributions are only an approximate model of real observational error, excellent as a starting point, in fact a gold standard for a starting point, but only part of the story. What we have here is "kurtosis".

Kurtosis (positive kurtosis, to be precise) is a ponderous name for a simple phenomenon in observations: you get more outliers than a pure Gaussian distribution would imply. And most people who have done observations with manual instruments are familiar with this phenomenon though they rarely have a name for it. For a navigation example, suppose you have a navigator who has a standard deviation of Sun altitude sights of 0.9 minutes of arc. That's not an unreasonable number. It implies that roughly two-thirds of observations (actually 68%) are within 0.9 minutes of arc of the truth. But the standard normal distribution tails off very rapidly. This means that the odds of finding an observation at three or four standard deviations away from the truth are extremely low --by this THEORETICAL model of the error distribution. Specifically, the odds of an observation at 3 s.d. with an error of +/-2.7 minutes of arc, or more, are about 1-in-370 --for a Gaussian normal distribution. The odds of an observation at 4 s.d. with an error of +/-3.6 minutes of arc or more are about 1-in-16,000. That number implies that you could shoot Sun altitudes five times a day, every day of the year, for over eight years and still only have an even-money chance of seeing an observation with an error of 3.6'. But that is not the reality of sextant observations. The normal distribution is a model with zero kurtosis. In the real world, at least from every practical set of observations that I have seen, the probability of points "in the tails" of the distribution are much higher. For example, you might get a 3.6 minute of arc error one time out of a hundred observations or even one in fifty, or in other words, with hundreds of times greater frequency than the standard normal distribution would imply. That's called "kurtosis" (for those who like even more arcane terminology, it is technically a "leptokurtic" distribution).

If you want to model observations that have kurtosis, there is an easy way to do it, and it has a direct relationship with the origins of these "outliers" in the real world. Generate random variables as follows: with some probablity f (e.g. 80%) take random numbers from a Gaussian normal distribution with a relatively small standard deviation. In the case here, we might take 80% of numbers from a normal distribution with standard deviation 0.7'. These correspond to normal "good" observations. For all other simulated observations (necessarily with probability 1-f, of course), take the observations from a Gaussian distribution with a significantly larger standard deviation, perhaps 3.0' in the case described here. These correspond to obsrevations where something has gone wrong but not at a level that we immediately detect. They're the sort of observations that we might occasionally mark down with a question mark or maybe just have a "funny feeling" about but they're not the sorts of observations that you would immediately throw it. The random numbers you will get from this "mixed" simulation will generally resemble normally distributed numbers until you look more closely at the statistics. This isn't magic. It's good science. If there's any life left in traditional navigation, there's every reason to seek modern methods of analysis. There's nothing wrong with trying to cull outliers in observational data when there is significant kurtosis.

-FER

----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------