# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Slide rule sight reduction accuracy**

**From:**Paul Hirose

**Date:**2009 Jun 15, 21:07 -0700

In a recent message I described a method to reduce celestial navigation observations with an ordinary slide rule using rectangular coordinates instead of spherical trigonometry. At the time I omitted some of the mathematics. Here is the full procedure. Convert LHA to an angle "theta" which is -90° (or 270) at the meridian, increasing east. This step isn't really necessary, but omitting it puts the rectangular coordinate frame into an unconventional orientation. It's no problem mathematically, but I find it awkward to visualize. theta = -90 - LHA Convert the body's local hour angle and declination to rectangular coordinates in a frame whose +z axis is directed to the north pole and +y axis directed to intersect Earth's axis. x = cos(dec) * cos(theta) y = cos(dec) * sin(theta) z = sin(dec) Rotate the coordinate frame about the X axis by the complement of latitude. This orients the +z axis to the zenith and +y north. In the second equation, y is the old y, not the new y computed in the first equation. y = y * cos(90-lat) + z * sin(90-lat) z = y * -sin(90-lat) + z * cos(90-lat) Find azimuth. Note that x and y are swapped from their usual positions so azimuth will be zero at north, increasing east. This formula yields a value in the range -90 to +90. If y < 0, add 180 degrees. az = arctan(x / y) Compute the body's distance from the z axis. r = sqrt(x*x + y*y) Compute elevation. el = arctan(z / r) I implemented this in a computer program which simulates slide rule accuracy. At each place a slide rule would be used, the result is multiplied by a number of the form (1 + x), where x is a random value, centered on zero, with Gaussian distribution and .001 standard deviation. In other words, the simulated slide rule has .1% accuracy. That's the figure commonly quoted for 10 inch slide rules, and in a test with one of my own rules I confirmed it. Sight reduction problems are automatically generated, starting with a random azimuth and elevation. In order to evenly distribute the targets about the sky, elevation is the arc sine of a random number between 0 and 1. (If you simply distribute elevations evenly between 0 and 90 degrees, the band of sky from 0 to 10 degrees will have as many targets as the band from 80 to 90, though the latter is much smaller.) A random latitude is obtained with the same arc sine method. The program can restrict elevations and latitudes to specified limits; I restricted elevations to 5 - 80 degrees and latitudes to 0 - 70. Declination and LHA are then computed from azimuth, elevation, and latitude. All these values are, for practical purposes, perfectly accurate. Declination, LHA, and latitude are submitted to the sight reduction routine, and the returned azimuth and elevation compared to the correct values. This occurs in a loop which runs any desired number of problems and tabulates the statistics. With this Monte Carlo simulation program I've found the slide rule sight reduction method outlined above is accurate in elevation to 3.1 minutes (square root of the mean squared error). About 95% of the results are within 6.2 minutes. The worst case results are about 15 minutes off. These appear to be due to unfavorable combinations of the random errors; I can't see any pattern in the azimuths and elevations where they occur. Azimuth RMS error is about 3.3 minutes. Worst cases are nearly one degree, and always occur when the problem is near the upper elevation limit (80 degrees in this test). My program is designed in a modular fashion so different sight reduction algorithms can be plugged in easily. I plan to implement others. If anyone has a burning desire to see a certain method put to the test, speak up. I'll move it to the head of the list. -- I filter out messages with attachments or HTML. --~--~---------~--~----~------------~-------~--~----~ Navigation List archive: www.fer3.com/arc To post, email NavList@fer3.com To unsubscribe, email NavList-unsubscribe@fer3.com -~----------~----~----~----~------~----~------~--~---