# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: A slope example**

**From:**Antoine Couëtte

**Date:**2010 Dec 1, 22:54 -0800

Dear Piterr,

In further reference to both :

- Your Post [NavList 14584] Re: A slope example, (with a part of it reproduced down below this post) and

- my preliminary reply in my Post [NavList 14589] Re: A slope example,

*******

I have retrieved my 'old' software giving me both :

- linear regression coefficient and

- comparison between observed and predicted geocentric slopes

and I have used it for our VEGA shots example.

*******

Here are my results :

1 - I see no typo in the data I published, :-))) and

2 - the predicted geocentric slope is -11.44 degrees per hour at the mean time of all 5 observations. This value takes everything in account, both body RA and Dec changes (if any) and observer's motion effects in both Latitude and Longitude.

3 - my Nav Logbook specifies "VERY BAD HORIZON ALL AROUND" (height of eye was 24 meters, i.e. close to 80 feet), and

4 - the single shots intercepts show as follows in NM ( far too many digits, you know ...) :

-2.510 -2.053 -1.599 -1.682 and -0.479 which certainly explains why you indicated that one of the sights - namely the last one - does not seem "in line with the others".

*******

If we work all 5 shots, we get :

Average value of the 5 intercepts : - 1.665 (i.e. AWAY)

Correlation coefficient : 0.999 , and

Observed geocentric heights slope : -10.97 degrees per hour (vs -11.44 computed)

*******

If we discard the last shot, we get :

Average value of the first 4 intercepts : - 1.961 (i.e. AWAY)

Correlation coefficient : 1.000 , and

Observed geocentric heights slope : -11.15 degrees per hour (vs -11.44 computed)

*******

DISCUSSION OF RESULTS :

Obviously the 4 shot fix seems might seem preferable (and even "much preferable" maybe to some people ?) to the 5 shot fix since it gives a better slope (no significant difference between both Correlation Coefficients)

Well ... many years of practice at sea have shown to me that maybe this 5 shot is - under current circumstances - probably as "valid" as the 4 shot intercept. Let me explain :

First of all, the correlation coefficient is one of the 2 required "tools" I used. From experience I have set up a "CC warning" only if CC inferior to 0.90 (this value did not show up at all in a recent post I published). So I had no alert from the "CC monitor" here.

Note : CC's are far from being sufficient. And your comments (down below) about CC's are very right since there IS a definite QUOTE "danger that outliers can significantly skew the derived slope provided by linear regression" UNQUOTE

So, having acknowledged this danger, how can we get around it ? As indicated in earlier posts, the solution I had been using was to also check observed slopes against predicted slopes. I had defined a "variable alerting threshold" as a function of (mainly) the overall duration of the Observations set. For our 3min23s overall duration example, the "threshold alert" is set to +/- 0.5 degree/hour between observed/predicted slopes difference. With an observed difference equal to 0.473 degree per hour in this case, I was still within limits. Therefore : no warning either about an outlier here.

*******

I am not claiming that a 4 shot approach is wrong, but certainly the 5 shot approach in no ways a NO-NO. Experience has repeatedly shown me to discard data, and especially averaged data only when I HAVE to.

Let us admit that this specific 5 th shot WAS definitely an outlier. Then just on Vega it brings up an intercept error equal to 0.3 NM. Since I took 14 shots in total on that evening (Moon, Venus, Jupiter and Vega at the end) the overall resulting error since narrows down to well under 0.1 NM.

You will then rightly observe (or even mutter ?) that increasing the number of observations will also increase the possibility for more outliers ... :-)))

On that evening, my overall fix was in Azimut 089 and at a distance of 2.2 NM from the LoranC fix, and the LOP's dispersion was 1.0 NM around observed fix.

Given the uncertainties of LoranC much less accurate than GPS to-day, and given the prevailing horizon, I was quite happy with my fix for that evening.

*******

This brings us back to a fundamental point earlier addressed. As we practice it, CELNAV is BOTH a definite SCIENCE, and also an ART. This is where your own criteria can come to play, obviously within the limits of science. The example here-above is a good one. The horizon was very bad all around, no warning came from either CC or slope, therefore I accepted all five shots.

Finally, and again from past personal experience, once the "Flashing Outliers" have been dealt with - and observing at regular intervals on a series of shots greatly helps here ... - I no longer resort to the full Correlation Coefficient/Slopes Comparison software. I have just retained the option of discarding (Averaged) LOP's once I have hand-drawn them on a chart.

*******

Dera Piterr, this pretty much concludes all my views on that single example.

Thank you for your Kind Attention and

Best Regards

Antoine

Antoine M. "Kermit" Couëtte

*****************************************************************************

From your post [NavList 14584] Re: A slope example

I guess one danger here could be an error creeping in - eg; from the data entry process - which may not be noticed, as one number is so very like another, is it not so? For example, once I plotted your "Averaged observation time : 16h30m11.060s, and Averaged Observed geocentric height 69d02.558m" it was immediately obvious that this point sat significantly below the very regular line of your sights, apart from apparently contradicting one of your sights taken close to this adopted time, a sight which appears in line with the others.

Linear regression is a function of the data points. They could be very good data points, as yours are in the example, and then the slope will approximate the actual slope. Or not, and then the derived slope may not approximate the actual slope at all. In other words, there is a danger that outliers can significantly skew the derived slope provided by linear regression. This has been pointed out by David Burch, together with an example, here:

----------------------------------------------------------------

NavList message boards and member settings: www.fer3.com/NavList

Members may optionally receive posts by email.

To cancel email delivery, send a message to NoMail[at]fer3.com

----------------------------------------------------------------