# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: HO 211 (Ageton) sight reduction accuracy**

**From:**Paul Hirose

**Date:**2016 Jun 22, 21:42 -0700

On 2016-06-11 22:08, Paul Hirose wrote: > alt alt alt az az az > RMS >.5' max RMS >.5° max > > 1.23' 14.6% 31.3 2.76' .184% 2.0° Ageton no interp > 1.26' 14.9% 31.5 2.87' .184% 2.2° no interp, whole numbers > .55' 8.8% 25.0' 2.25' .111% 2.1° interp all > .57' 12.8% 25.6' 2.65 .170% 1.9° interp K<8 8.5% > .59' 13.6% 25.4' 2.65 .168% 2.1° interp K<5 5.3% > .58' 12.8% 24.6' 2.68' .174% 2.1° interp t<8 7.3% > .59' 13.5% 24.4' 2.65' .171% 2.0° interp t<5 4.4% > .68' 14.0% 24.9' 2.82' .195% 2.3° interp t<5, whole numbers Standard Ageton tabulates the A and B functions every 0.5'. If that is changed to 0.2', there's a noticeable accuracy increase. If you don't interpolate, root mean square altitude error improves from 1.23' to 0.69', and the sights with altitude errors greater than 0.5' decrease from 14.6% to 4.2%. Interpolation of B(R) when K or t is near 90 makes only a marginal improvement to those numbers. In the standard table, the precision of function A increases from whole numbers to tenths at 84°. If we change that to 54° (with a corresponding change to B), plus 0.2' tabulation interval, and interpolate B(R) when t is within 5 degrees of 90, RMS altitude error is only 0.28', with 98% of altitudes within 0.5' of the truth. Increasing the size of the danger zone from 5° to 8° doubles the number of sights to interpolate (from 4.3% to 8.6%), with almost no change in the statistics. Likewise using K instead of t as the basis of the danger zone. As I explained in an earlier message, t is less work for the navigator than K. Below are my statistics. As before, the randomly generated test problems are limited to altitudes from 5° to 80° and latitudes from 0 to 70°. Columns are altitude root mean square error, percentage of sights with more than 0.5' altitude error, max altitude error detected during a Monte Carlo simulation run of 100,000 test points, azimuth RMS error, percentage with more than 0.5 degree azimuth error, max azimuth error. The last column shows the characteristics of the simulated table, and the interpolation criterion. For example, "interp t<5" means interpolate B(R) when t is within 5° of 90°. Precision of the tabulated A values changes to tenth at either 84° or 54° Table interval is 0.2' throughout, except the last line. alt alt alt az az az RMS >.5' max RMS >.5° max table type 0.69' 4.24% 23.7' 1.71' 0.054% 1.457° 0.2' 84° no interp 0.55' 3.74% 21.1' 1.65' 0.05% 1.445° 0.2' 84° interp t<5 0.55' 3.74% 23.8' 1.62' 0.047% 1.102° 0.2' 84° interp K<8 0.62' 2.96% 19.3' 1.41' 0.034% 0.852° 0.2' 54° no interp 0.28' 1.95% 12.3' 1.30' 0.027% 1.045° 0.2' 54° interp t<5 0.27' 1.52% 12.5' 1.32' 0.027% 1.015° 0.2' 54° interp t<8 0.27' 1.65% 12.0' 1.26' 0.021% 0.787° 0.2' 54° interp K<5 0.28' 1.42% 12.1' 1.30' 0.023% 0.987° 0.2' 54° interp K<8 0.42' 11.5% 14.2' 2.53' 0.16% 1.650° 0.5' 54° interp t<5 I think the optimum "HO 211+" method combines a 0.2' tabulation interval with interpolation of B(R) when t is within 5° of 90°. Of course the table would be larger: 90 pages vs. 36. That's still not very big, and could be cut in half by "turning around" at 45°, as in the Bayless table. The price you pay is that's it's easier to make a blunder, as the A and B columns exchange places depending on the angle. What if you tabulate A and B values every tenth minute? Stay tuned.