# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: HO 211 (Ageton) sight reduction accuracy**

**From:**Paul Hirose

**Date:**2016 Jun 28, 22:57 -0700

On 2016-06-22 22:03, Paul Hirose wrote: > alt alt alt az az az > RMS >.5' max RMS >.5° max table type > > 0.69' 4.24% 23.7' 1.71' 0.054% 1.457° 0.2' 84° no interp > 0.55' 3.74% 21.1' 1.65' 0.05% 1.445° 0.2' 84° interp t<5 > 0.55' 3.74% 23.8' 1.62' 0.047% 1.102° 0.2' 84° interp K<8 > > 0.62' 2.96% 19.3' 1.41' 0.034% 0.852° 0.2' 54° no interp > 0.28' 1.95% 12.3' 1.30' 0.027% 1.045° 0.2' 54° interp t<5 > 0.27' 1.52% 12.5' 1.32' 0.027% 1.015° 0.2' 54° interp t<8 > 0.27' 1.65% 12.0' 1.26' 0.021% 0.787° 0.2' 54° interp K<5 > 0.28' 1.42% 12.1' 1.30' 0.023% 0.987° 0.2' 54° interp K<8 > > 0.42' 11.5% 14.2' 2.53' 0.16% 1.650° 0.5' 54° interp t<5 I forgot to examine a couple cases in my previous message on a 0.2' table. If such a table is used with no interpolation and you exclude sights where t is within 5° of 90, and also exclude stars with declination greater than 75°, root mean square altitude error is 0.17', 1.25% of the solutions have more than half a minute error, and the worst is 3.1'. In addition to the above, suppose the precision of function A changes from whole numbers to tenths at 54° instead of 84°, and there's a corresponding change to B. Then RMS altitude error decreases to 0.14', 0.648% exceed 0.5' error, and the worst altitude is 1.7'. The significance of 54° is that it's the point where the A function no longer changes one count per tenth minute difference in angle. Now consider a hypothetical HO 211 tabulated every tenth minute. With 600 entries per degree, I don't think it would be practical, but it may be interesting to see how such a table performs in a Monte Carlo simulation. With no interpolation, altitude RMS error is 0.60', 3.24% exceed 0.5' and worst altitude is 21.2' off. Moving the change of precision to 54° makes a lot of sense at this tabulation interval, since otherwise you get many adjacent table entries with the same values. That improves altitude RMS from 0.60' to 0.36', but there's little difference in the worst error. Increasing precision to two decimal places at 85° doesn't help much either. Interpolation of B(R) from A(R) when t is near 90° gets RMS altitude error down to a quarter minute. However, the biggest gain occurs if we simply exclude t near 90 and high declinations. Then RMS altitude error is 0.08' and the worst sight is only 1.1'. Of course it's not always practical to exclude the 8% of sights that meet those criteria. And as I've shown, interpolation when a sight is in the danger zone doesn't guarantee accurate results. We need a different solution for such sights. One is the Sadler technique, which will be evaluated in my next message. alt alt alt az az az RMS >.5' max RMS >.5° max table type 0.17' 1.25% 3.1' 1.65' 0.05% 1.362° .2' 84° no interp t<5 d<75 0.14' 0.648% 1.7' 1.32' 0.022% 0.882° .2' 54° no interp t<5 d<75 0.60' 3.24% 21.2' 1.43' 0.037% 1.153° .1' 84° no interp 0.36' 1.70% 18.2' 0.85' 0.006% 0.652° .1' 54° no interp 0.25' 1.30% 14.4' 0.79' 0.006% 0.773° .1' 54° interp t<5 0.38' 1.71% 14.6' 0.86' 0.002% 0.733° .1' 54°/85 no interp 0.26' 1.18% 12.4' 0.82' 0.004% 0.808° .1' 54°/85 interp t>5 0.08' 0.14% 1.1' 0.85' 0.002% 0.672° .1' 54° no interp t<5 d<75