NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: Rejecting outliers: was: Kurtosis.
From: Geoffrey Kolbe
Date: 2011 Jan 01, 16:58 +0000
Peter Fogg wrote:
Thankyou Peter, and a good morning to you.
Actually, that diagram is a very good example of what I am trying to get across. On seeing that data, my first question would be, "Are my sights usually such a good fit to the line?" In other words, is the standard deviation on the random error of my altitude data usually as small as indicated by the four points that sit very close to the line?
If the answer is yes, then there is very little doubt that the orphan point up at the top does not belong to the data set that sits so nicely on the line. It is so far out that I would be pretty certain that there was some difference in the circumstances that prevailed for that measurement against the four others.
When taking a run of sights like that, I find that a steady rhythm is established, so it is fairly easy to pick out errors in timing. If the altitude was off by a whole degree then an error in reading the sextant would seem likely. Because of this, I would be particularly suspicious of the fact that the orphan data point is not the first or the last in the run of data.
But if those and the other usual suspects are discounted, then it starts to become increasingly likely that there is some "systematic" problem, like a loose index mirror on the sextant which, for one of the sights, moved slightly in its seating. Now, if something like that was the problem - as would seem likely with the outlier so far out - we suddenly have good reason to mistrust all the data.
Your response, I gather, would be to discard the outlier and carry on as though it never existed.
Fine, I have said what I needed to say.
Regards,
Geoffrey Kolbe
From: Geoffrey Kolbe
Date: 2011 Jan 01, 16:58 +0000
Peter Fogg wrote:
I've added some simple diagrams to text to try and show that its the other way around; apparent outliers tend to reject themselves because they stubbornly refuse to join a 'line of best fit', but whereas in the past I've been able to place text and diagrams directly here I can't this morning, for reasons unclear.
Therefore the next least-worst alternative has been adopted, and I've attached my reply as a PDF file. Hope you can open it. Its only about 60KB, thus a file of modest size.
Thankyou Peter, and a good morning to you.
Actually, that diagram is a very good example of what I am trying to get across. On seeing that data, my first question would be, "Are my sights usually such a good fit to the line?" In other words, is the standard deviation on the random error of my altitude data usually as small as indicated by the four points that sit very close to the line?
If the answer is yes, then there is very little doubt that the orphan point up at the top does not belong to the data set that sits so nicely on the line. It is so far out that I would be pretty certain that there was some difference in the circumstances that prevailed for that measurement against the four others.
When taking a run of sights like that, I find that a steady rhythm is established, so it is fairly easy to pick out errors in timing. If the altitude was off by a whole degree then an error in reading the sextant would seem likely. Because of this, I would be particularly suspicious of the fact that the orphan data point is not the first or the last in the run of data.
But if those and the other usual suspects are discounted, then it starts to become increasingly likely that there is some "systematic" problem, like a loose index mirror on the sextant which, for one of the sights, moved slightly in its seating. Now, if something like that was the problem - as would seem likely with the outlier so far out - we suddenly have good reason to mistrust all the data.
Your response, I gather, would be to discard the outlier and carry on as though it never existed.
Fine, I have said what I needed to say.
Regards,
Geoffrey Kolbe
