# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: How good is St. Hilaire?**

**From:**George Huxtable

**Date:**2010 Mar 2, 09:50 -0000

In reply to my comment about the presentation of multi-LOP results by the AstronavPC procedure- "The problem is that [...] the confidence-ellipse, is based on only a single sample. It's relying on the statistics of a single event. " Frank commented- "Right. But I wouldn't describe this as a fundamental flaw. It's just a particular choice in the way they're calculating the confidence ellipse." =============== If you calculate the confidence ellipse from the way that three LOPs intersect, AND NOTHING ELSE, then there is no other choice. And as a result, there is the significant chance that they will intersect in such a way as to give a seriously misleadingly-small answer for the confidence ellipse. The answer, of course, is to introduce additional information, and a dash of common sense. To treat each position line as being surrounded by an error-band, just as Frank suggests. And if a computer-analysis program for multiple sights could do just that, all would be well. But what I was pointing out was that the error-ellipse was indeed calculated, by that program, without reference to any such commonsense notions. As a result, significantly often (from a three-sight cocked-hat), it will so happen that the user will be presented with a confidence-ellipse that bears no relation to the expected scatter in the deduced position. If the user takes that ellipse, drawn enticingly on each plot, at face value, and takes it to really mean what it looks as if it means, he will be badly misled. And could be led into danger. And there is no warning provided about the matter, in that document, or (as far as I can tell) on the computer screen. That is the fundamental flaw. George. contact George Huxtable, at george@hux.me.uk or at +44 1865 820222 (from UK, 01865 820222) or at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK. ----- Original Message ----- From: "Frank Reed"To: Sent: Tuesday, March 02, 2010 2:30 AM Subject: [NavList] Re: How good is St. Hilaire? George, you wrote: "The problem is that [...] the confidence-ellipse, is based on only a single sample. It's relying on the statistics of a single event. " Right. But I wouldn't describe this as a fundamental flaw. It's just a particular choice in the way they're calculating the confidence ellipse. The way the analysis is done, as presented, is by treating the standard deviation of the observations as an *output* of the statistical analysis. But you can also treat it as an input. And in fact, when I've coded this up for myself, that's how I do it. Consider the case of a single LOP. We can put a confidence "band" along that line with a width based on previous experience with that instrument and that observer. For example, we might have some data from the previous week's worth of fixes. We use that to "smudge" the LOP and give it some thickness. Everybody does this, even if it's just a mental adjustment. And if a confidence "band" doesn't sound very confident, then, of course, if we also have a DR position with some confidence limits around it, then we can combine those two and get a long and thin confidence ellipse where the "smudged" LOP overlaps the DR confidence ellipse (I assume most navigators do this part in their heads). With a two-body fix, we can draw a confidence ellipse again based on the standard deviation of observations as an input. It's an ellipse about the same size as the "overlap box" for the error bands around each LOP. And with a three-body fix, this answers your concern about a confidence ellipse being drawn too small when the LOPs just happen by chance to coincide in a small triangle. That small triangle should not imply a small confidence ellipse, and it doesn't if the s.d. is an input rather than calculated from the observations themselves. Eventually we get to cases with larger numbers of observations. For example, there's the case that I call a "rapid-fire fix" where an observer is shooting one body (presumably the Sun) every few minutes over an extended period of time. Or we might be interested in an automated system that is shooting the stars several times per second. In these cases where the number of observations is much greater than one, it does make sense to treat the standard deviation as an output of the data analysis. This observed s.d. might not match our input s.d. perhaps because there's a hazy horizon or some other factor making conditions significantly worse than expected or occasionally better than expected. An automated system with a large number of observations would have a confidence ellipse that fluctuates in size as observational conditions change. This would be similar to the fluctuating size of the confidence ellipse around a GPS ellipse as the number of available satellite signals changes. There's probably some ideal N of observations where the calculated standard deviation is better than an input standard deviation. I will vote for five. If we have fewer than five observations, use an input s.d. for the error ellipse calculations. If more than five, use a weighted average of some sort. After ten or twenty observations, the observed s.d. should dominate. There's no reason to treat that publication as a bible, right? Re-derive its math from first principles, and try out the different ways of treating the s.d. By the way, back to that bit I mentioned about combining a celestial LOP with a DR error ellipse. Suppose we have two LOPs with a very low angle between them. The confidence ellipse around them is rather long and thin. If we ALSO have a confidence ellipse around a DR position, derived in some meaningful way, then the best estimate of position lies in the area where those ellipses overlap, not at the center of the celestial fix. That, it seems to me, is one area where traditional navigation does not have any systematic rules in place, despite the fact that this case is open to relatively simple statistical analysis. Instead it's just assumed that the navigator will figure it out by intuition or "common sense". -FER ---------------------------------------------------------------- NavList message boards and member settings: www.fer3.com/NavList Members may optionally receive posts by email. To cancel email delivery, send a message to NoMail[at]fer3.com ----------------------------------------------------------------