NavList:

Since statistics were never my area of study, perhaps I would do better to provide my methodology and then raw data to let you play with the math and get more accurate numbers than I came up with.

 

First the fixes themselves.  This was my most comprehensive set of data points for finding errors of position between the experimental position (celnav fix) and my datum (GPS fix) that I have ever gathered.  I am assuming here that my GPS is accurate to about 20 meters or so, or less than the beam of my ship in general.  There might be some argument here, but the distances even to 100 meters isn't really significant for the kind of errors I am getting.

 

During various twilights, both AM and PM, I shot a number of sights of various bodies and entered them into my Skymate pro software.  The software required body, time, and Hs.  With the givens (IC, Dip, T/P, etc) already entered, and a GPS fix saved to the second of time on the same GPS receiver every time; as the "DR"  The program then computed the AZ/intercepts and advanced/retarded the lines to give a calculated star fix.  This fix was compared to the GPS position via mathematical sailings on my NC-2 calculator.  The resulting distance was my "fix error."  I then compared my fix error for each set of sights and came up with my percentages for each range of fix error.  I found that of the 70 fixes, I got 8 fixes with 1 nm or more error, which is 11.4%, or about 88.6% of my sights below 1 nm of fix error.

 

What i cannot explain is how the program derives the fix position from the scattered lines, but it certainly isn't the center of any cocked hat.  There is some statistical analysis going on there, but I have no knowledge of what methods are being used.  All I know is that if the conditions were good, then I was able to confirm my position within 1 nm a majority of the time. 

 

As for the intercepts individually, I just ignored whether they were towards or away and took the intercept calculated by my program for each line and then used the MS Excel function STDEV on each column of numbers to arrive with the 0.8 for the lot of them, and the others in turn.  This is probably flawed, but without an in-depth study of Statistics, I don't really know how to do it otherwise.

A quick look at one of my 0.0 fix error sights gives me the following intercepts:

 

Jupiter  0.4 Twds 119.0 deg

Deneb 0.3 Twds 045.6

Altair 0.4 twds 107.4

Nunki 0.3 away 160.7

Antares 0.1 twds 202.4

Spica 0.5 twds 250.4

Alkaid 0.3 twds 316.

 

So we have 0.0 fix error, and a bit of error on each star.  I do not know why the computer got less than 0.05 error out of these flawed sights, but it did.

 

 I am not really into the math side of this discipline and merely know enough of it to get position et al, and recognize when things are going wrong and perhaps how to get myself back on course.

 

I am attaching a MS Excel workbook with two sheets attached.

 

The first sheet has the fix error of each sight on the left with a serial number.  This is also graphed on the right part of the sheet.  On the bottom I have used MS Excel formulae to determine average and standard deviation for the fix errors.  Please remember that these are only for the star fixes and don't include data from sunlines which are included in the second sheet.  Basically we are looking at the stars, planets, and a selection of moonlines.

 

The second sheet has the raw intercepts listed and the STDEV on the top of each column for all sights taken during the trip.  The right-most column is just a serial number for the sights and has nothing to do with collected data.  Perhaps the more mathematically inclined people can work some magic with them.  If you need to know whether they were "to" + or "away" -, I will have to redo the list when i have a bit more time.  I am on the ship at the moment, so it will have to wait until my schedule permits me some time to look them over.

 

Jeremy

Jeremy, I am trying to make sense out of your fix and LOP statistics.

There is an inconsistency between your LOP sigma of .800225 NM and the statistics for the fix accuracy. One sigma will only contain 39.35% of the fixes but you show 60% of the fixes within only 0.5 NM, which is .62 sigma, which should contain only 17.7% of the fixes. It should take 1.354 sigma to contain 60% of the fixes so should be 1.08 NM instead of the the 0.5 NM that you state.

Similarly, you show 1.0 NM, which is 1.25 sigma, containing 88.6% of the fixes but 1.25 sigma should only contain 54.2%. It takes a circle of 2.084 sigma to contain 88.6% of the fixes so should be 1.67 NM instead of the 1.0 NM that you state.

Looking at the discrepancy from the other direction, if 88.6% of the fixes were contained within just 1.0 NM (which means that 1.0 NM equals 2.084 sigma) then sigma for the LOPs must have been only .481 NM so there must be an error in the .800225 NM value that you stated.

So either my math or your math (maths for our English friends) must be in error (and I am not ruling out that it might be mine.) I am attaching an excerpt from Bowditch that explains the statistics. Note, to convert table Q7d to sigma from CEP just multiply each value by 1.177 which is the ratio between these two measures. You can use the given formulas to compute different probabilities and sigmas.

gl

------------------------------------------------------------

Jeremy wrote:

'll quote myself here [NavList 9887]:

On Sep 23, 2009 3:40 pm, Anabasi...---com wrote:
> I shot 70 star fixes on my trip from Japan to the USA and here are a few
> interesting statistics.
>
> 1) average fix error from GPS: 0.566 nm (low of 0.0 and high of 2.7 nm
> error)
>
> 2) 88.6% of the fixes were under 1.0 nm fix error.
>
> 3) 60.0% of the fixes were under 0.5 nm fix error
>
> 4) two fixes had errors under 0.05 nm (under 100 meters).
>
> My next project is to determine the average intercept based on the GPS fix
> as the DR but that is over 700 data points so might be awhile.
>
> Jeremy

As you can see, these are position errors, not LOP errors. I also ran standard Dev for my fixes and it was 0.5813 nm. I will add that my worst fix was 2.7 nm off under awful conditions.

The trouble I have with LOP statistics is that there are so many variables at play as render them only interesting but not very useful. With a good sky and horizon I can pinwheel every time (intercepts well under 1 nm), if the horizon and/or sky is terrible, I can have errors of several miles. Also, can someone expect the aforementioned statistics in a bouncing sailboat as opposed to a large ship? Probably not.

I just ran the statistics for the individual LOPs, 551 in all. These were shot in a wide variety of conditions including the haze of the East China Sea, near Cape Hope, and in the tropics. There is a wide variation of error from day to day and twilight to twilight depending on the environment.

I only ran them on the computer reduced sights as these use GPS fixes for the AP as opposed to my tabular reductions. (planets, the moon, and stars) I didn't use the computer for most of my sunlines, but there are a few included in there.

Standard deviation for all 551 sights was .800225.

I did shoot 1 moonlight fix. The position error was about 2.5 nm which would be good enough in a pinch, but well below my standard.

Jeremy
----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------

File: