# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Bygrave formula accuracy on 10 inch slide rule**

**From:**Paul Hirose

**Date:**2009 Jul 06, 15:19 -0700

My computer simulation of the Bygrave sight reduction formulas, worked on a 10 inch slide rule, had altitude accuracy of 1.8' and azimuth accuracy of 2.0'. Those are the square roots of the mean squared errors. In a run of 500,000 random sight reduction problems, 95% of the solutions were correct within plus or minus 3.7' in altitude, and 95% were within plus or minus 4.2' in azimuth. The maximum altitude error seen by the program was 10.4'. Worst cases always seem to occur around 40° - 50° altitude. Note that the Bygrave solution reads altitude on the tangent scale, which is most compressed at 45°. The maximum azimuth error seen by the program was 28.0'. Worst cases occur at high altitudes. I was suspicious of the accuracies reported by the test program. They seemed too good, so I worked six random problems (generated by the program) by hand on a 10 inch rule. Altitude errors (minutes) were +1.1, -5.2, 0.0, +1.1, -2.3, -.6. Azimuth errors were -.8, +.1, +.3, -1.3, +1.1, -1.4. These results suggest the program's modeling of slide rule errors is realistic. To operate the slide rule I wore reading glasses but did not use my hands free magnifying glass, though it would have helped a good deal. My program generates each sight reduction problem from a random azimuth and altitude, the latter being weighted so the simulated stars tend to have constant density everywhere in the sky instead of packing closer with increasing altitude. Altitudes less than 5° or greater than 80° are rejected. A random observer latitude between 0° and 70° is generated in similar fashion. Each azimuth, altitude, and latitude combination is converted to LHA and declination. The sight reduction module converts these values back to azimuth and altitude, injecting a random error in each slide rule operation, then compares results to the correct values. Slide rule error is assumed to be .1% RMS per multiplication or division (which involves two settings and one reading). The Bygrave azimuth formula requires *three* settings and one reading, so for that calculation I increase the error accordingly. I have modeled the Bygrave formulas on a standard slide rule, but not the Bygrave rule itself. Its error should be in inverse proportion to its scale length relative to a 10 inch rule. -- I filter out messages with attachments or HTML. --~--~---------~--~----~------------~-------~--~----~ Navigation List archive: www.fer3.com/arc To post, email NavList@fer3.com To unsubscribe, email NavList-unsubscribe@fer3.com -~----------~----~----~----~------~----~------~--~---