NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
From: Peter Fogg
Date: 2005 Jan 12, 16:29 +1100
"I don't see how he has
calculated that number"
There are six chances that a
number rounded to, say, 24, could have been rounded from: 23.5, 23.6, 23.7,
23.8, 23.9, 24.0 - so one chance in six that the number was 23.5. As there are
6 values in the sight reduction table, the probability of a maximum error of
3.0 (6 numbers of something.5 each) is 1 in 6 to the power of six, 0.000021433
expressed as a decimal. Thus a negligible chance. Quoting a remote possibility
of error as a major shortcoming to be taken seriously is what I understand by
disingenuous - just to further merge not so very dissimilar topics.
"We have just agreed
that the multiple roundings in the calculation we have considered are unlikely
to exceed 2'. Even so, that could imply that the true position may not be in
the unique 1' rectangular box that was the stated result of the calculation,
but could quite likely be in one of the four adjacent 1' boxes that surround
it, and with a smaller chance, in one of the adjacent 1' boxes that surround
that. That was just the result of the agreed rounding errors in the calculation
process, and errors in the observation itself haven't even been considered
yet."
Oh dear, George, you need to
read the argument carefully before rushing into print.
" ...even if the fix
is entirely accurate). It means that the position is somewhere within a
rectangle..." to quote myself.
We are assuming that this
fix is accurate. Being accurate means that the position is somewhere within the
rectangle indicated by the fix.
"I don't follow what
Peter is saying here (or if I do, then I disagree)."
Wonderful stuff! Disagree
first, understand later!
-----Original Message-----
From: Navigation Mailing
List [mailto:NAVIGATION-L@LISTSERV.WEBKAHUNA.COM]
On Behalf Of George Huxtable
Sent: Wednesday, 12 January
2005 12:32 PM
To:
NAVIGATION-L@LISTSERV.WEBKAHUNA.COM
Subject: Re: On potential
error introduced by rounded values
There are two separate
topics merged into Peter Fogg's recent pair of mailings, for both of which
George Bennett's "Complete on-board Celestial Navigator has provided
examples.
The topics are
1. The effect of multiple
roundings on precision of the result, which I will respond to here under the
threadname "On potential error introduced by rounded values."
2. The defects of the
formula for obtaining azimuth from its sine and the errors that it gives rise
to in Bennett's lookup table for azimuth, which I'll deal with elsewhere under
the threadname "Sight Reduction Formula Question".
I wrote-
>"If we take 6 such
roundings, then
>
>in theory the maximum
amount they could possibly (but most unlikely)
>
>contribute to an error
in position is ±3 miles..."
and Peter Fogg replied-
>And the chance of that
most unlikely event is one in six to the power
>of six, or 0.00002, or
0.002%?
I don't see how he has
calculated that number, and perhaps Peter will explain. The probability of the
rounding errors combining to exceed ±3 miles is precisely zero. And the
probability of the error being exactly 3 miles is also zero.
I think that there's a
conceptual problem here. For a contiuous distribution such as we're discussing,
the probability of the error being ANY exact number is always zero. You have to
express it as the probability of being in some bracket between two values, or
exceeding some limit, for the probability to have meaning.
>If this is correct, the
results from models similar to the Excel sample
>are unlikely to show an
error result of more than 2 minutes of arc in a
>significant number of
cases.
I don't think it is correct,
but even so, I agree with that conclusion.
> When a
>fix is expressed to
whole minutes of arc it doesn't mean the boat is at
>that intersection (it is
almost certainly not, even if the fix is
>entirely accurate). It
means that the position is somewhere within a
>rectangle (a square at
the equator) bounded by the halfway points to
>the next intersection of
minutes of arc of lat/long.
I don't follow what Peter is
saying here (or if I do, then I disagree). The true position isn't necessarily
in that one rectangle. You could only state with certainty that the true
position was in that one 1' rectangle if you knew that the fix was in itself
entirely accurate, in terms of both observation and calculation, except for
some final single rounding operation that expressed the result to the nearest
whole minute.
We have just agreed that the
multiple roundings in the calculation we have considered are unlikely to exceed
2'. Even so, that could imply that the true position may not be in the unique
1' rectangular box that was the stated result of the calculation, but could
quite likely be in one of the four adjacent 1' boxes that surround it, and with
a smaller chance, in one of the adjacent 1' boxes that surround that. That was
just the result of the agreed rounding errors in the calculation process, and
errors in the observation itself haven't even been considered yet.
George.
================================================================
contact George Huxtable by
email at george@huxtable.u-net.com, by phone at
01865 820222 (from outside
================================================================
