# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Accuracy of sextant observations at sea (Mr David Burch's ARTICLE on "Averaging Celestial Sights")**

**From:**Antoine Couëtte

**Date:**2010 Dec 4, 22:38 -0800

Dear Piterr,

As regards Accuracy of Sextant observations at sea,

you have provided an interesting study by Mr David Burch with bears the title "Aver[a?]ging Celestial Sights".

It is my understanding that in his study Mr Burch advocates the use of the "predicted slope" vs "observed" slope. I can certainly follow him 100 % here. It is a very good sense suggestion.

I am now referring to the two graphs given in his paper :

- the first one on page 2 (data plotting + 1 approximation line) which I will subsequently refer to as "GRAPH 1" , and

- the second and last one on page 5 with the following subtitle "calculated slope moved to data", which I will subsequently refer to as "GRAPH 2".

Again, I am not hesitating to highly value and fully support M. Burch's viewpoint of using "predicted" vs "observed" slopes.

Note : "predicted" means : computed/calculated whilst taking in account both the body position relative to the Observer's position combined to the Observer's motion itself.

However in this article - and as one can easily infer from both drawings - I am observing that THE APPROACH M. BURCH IS USING IS NOT A LEAST SQUARE APPROACH. HIS CHOSEN APPROACH ESTABLISHES AN A PRIORI DE FACTO DISCRIMINATION ON SOME OF THE DATA, AND IT ALSO GIVES AN A PRIORI HIGHER IMPORTANCE TO OTHER DATA.

In GRAPH 1, the drawn line is "TOO LOW" if compared to a "least square line". As a consequence, observations 1, 3 and 4 are given "100% credit" since the "fitting line" just crosses all three of them while Observation 2 is discarted 3 Miles away.

In Graph 2, while having a more accurate slope per se (and a "much better one" since it is the predicted slope) the final line has been drawn "TOO HIGH", i.e. too close from Observations 2, 3 and 4 and it now gives the impression that Observation 1 is now ... FIVE miles off !

NOT USING A LEAST SQUARE METHOD IS EQUIVALENT TO NOT TREATING ALL DATA EQUALLY JUST FROM THE ONSET. Once "flashing outliers" have been removed, only AFTER every single remaining data has been treated equally first, can we then take a personal decision to discard/remove/give less importance to one (or more) specific information.

There are times when giving an "a priori" unequal weight to some data can be fully justified (e.g. from personal experience when a sight had to be performed through the vessel engines smokes and fumes which rendered the horizon "fuzzy/blurry/uncertain"), but in the quoted article nothing indicates that such is the case.

So the conclusions drawn by M. Burch would certainly be less strong than what he claimed them to be simply because some data are not given "equal opportunity" from the onset, as can be seen from the displayed data fitting lines.

May I then respectfully invite Mr David Burch - if he reads me or if he can be reached one way or the other - to recompute his example through :

- using the standard least square method which will yield both an "observed" slope as well as standard deviation of data around this observed "regression line" defining the "data scatter" around this line, and

- a (definitely better) least square approach in which the slope is forced into its predicted value. It is my guessing that through this method, Observation #1 will certainly NOT show as being 5 miles off.

And I would also guess that the averaged values of both sets of processed information (i.e. the data processed without forcing the slope, and the data processed through forcing the slope into its predicted value - which again is a MUCH BETTER APPROACH) would now be quite close from one another. If no other requirement (azimut considerations as earlier mentioned) this is also ONE VERY GOOD REASON for using averaged values instead of any other point on the data fitting lines (whatever "observed" or "predicted" slope they might have) . Final note : personal experience has taught me that sets of 5 observations (when achievable) will give significantly more reliable results than sets of 4 observations.

To Mr David Burch, and if you read me, YES ! - and again - using a predicted slope is definitely superior to using a plain observed slope.

Dear Piterr, Thank you for your Kind Attention

Antoine

Antoine M. "Kermit" Cou�tte

********************************************************************************

[NavList] Re: Accuracy of sextant observations at sea

From: piterr11---com

Date: 1 Dec 2010 19:40

All those interested in this topic of accuracy of sextant observations, whether at sea or elsewhere, may be interested in the contents of this paper:

http://www.starpath.com/resources2/sight_average.pdf

----------------------------------------------------------------

NavList message boards and member settings: www.fer3.com/NavList

Members may optionally receive posts by email.

To cancel email delivery, send a message to NoMail[at]fer3.com

----------------------------------------------------------------