# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**On potential error in azimuth tables**

**From:**Peter Fogg

**Date:**2005 Jan 14, 11:25 +1100

George Huxtable wrote: "I don't remember seeing that work elsewhere. Was it posted to Nav-L? No warning, or analysis, about azimuth errors seems to exist on the Bennett website." I'm sorry George, hadn't realized it was necessary to warn you in advance before presenting any new works. You did send out a written invitation: "... if only to tweak Peter Fogg's tail a little, I can't resist pointing out that earlier criticisms of the Bennett tables revolved around the major errors in azimuth, of many degrees..." I guess natural modesty prevents you from revealing that these 'earlier criticisms' of 'major errors? came only from George Huxtable (as did the assertion about rounding errors accumulating in sight reduction, apparently now revised). And presume that these same qualities forbid you from making any substantive answer about the azimuth tables. " I would like to read the full text rather than extracts. Where can I do so, Peter? I would like to understand Bennett's analysis methods." On the website http://www.netspace.net.au/~gbennett/ you will find an email forwarding facility. I have always found George Bennett very responsive and helpful. To be fair, you have acknowledged, in answer to my query 'Sounds great. When can we expect to see a production model? Could this method be turned into a simple 'look up' table?' about a proposed method of azimuth determination: "I doubt if it could. It would need someone much more clever than I am. That is the one advantage of the sin Az procedure; it lends itself to a simple lookup table, such as Bennett has devised." Thank you, for that much at least. This is what I meant by the advantages outweighing the shortcomings in the vast majority of cases. It seems to me that there are now 2 possibilities. One is that the result from 15 million samples of use of the azimuth tables is incorrect. The other is that the ?major errors?, so harped upon and gloated over, are so remote in occurrence as to be negligible in effect*. In that case a retraction, at the very least, seems to be called for. (A small correction of my own: I should have said, when simplifying the results, that about 94% of about 15 million samples showed an error of azimuth of one degree or less, about 5% showed an error of two degrees or less, and that greater error than that occurs in a little more than 1% of cases. The table supplied contains the exact figures, to 4 decimal places) *How many lifetimes of using these tables every day would be required to come across errors that occur twice in 15 million goes? About five hundred lifetimes, given a 40 year non-stop nautical career. If my figures are correct. Am still very much only an amateur statistician. -----Original Message----- From: Navigation Mailing List [mailto:NAVIGATION-L{at}LISTSERV.WEBKAHUNA.COM] On Behalf Of George Huxtable Sent: Wednesday, 12 January 2005 12:32 PM To: NAVIGATION-L{at}LISTSERV.WEBKAHUNA.COM Subject: Re: Sight Reduction Formula Question There are two separate topics merged into Peter Fogg's recent pair of mailings, for both of which George Bennett's "Complete on-board Celestial Navigator has provided the examples. The topics are 1. The effect of multiple roundings on precision of the result, which I will respond to under the threadname "On potential error introduced by rounded values." 2. The defects of the formula for obtaining azimuth from its sine and the errors that it gives rise to in Bennett's lookup table for azimuth, which I'll deal with here under "Sight Reduction Formula Question". I wrote earlier- "There's a third option, that for some reason doesn't find its way into >many textbooks. Get the azimuth from its tan! This formula is- Tan Z = >sin >(hour-angle) / (cos (hour-angle) sin lat - cos lat tan dec) and the >rules for putting Z into the right quadrant, 0 to 360, clockwise fron >North, are- If tan Z was negative, add 180 deg to Z. >If hour-angle was less than 180 deg, add another 180 deg to Z." I should perhaps have added (which ought to be obvious), if the To which Peter Fogg replied- "Sounds great. When can we expect to see a production model? Could >this method be turned into a simple 'look up' table?" ================ I doubt if it could. It would need someone much more clever than I am. That is the one advantage of the sin Az procedure; it lends itself to a simple lookup table, such as Bennett has devised. Bennett's table provides a quantity X, which is sin B cos C. In the first lookup operation, the rows are regarded as LHA and the columns as Dec, so this provides X = sin LHA cos dec. In the second operation, the rows are of the azimuth that will result and the columns are Altitude, so this provides X = sin Az cos Alt. And the double lookup operation gets to a solution for Az by equating the two values of X, so sin LHA cos dec = sin Az cos Alt, or Sin Az = (sin LHA cos Dec) / cos Alt which was what we were after. The errors result from an unfortunate combination of using the arc-sine function (which requires very precise numbers to work from, when angles are near East or West) with the rounding of all quantities to the nearest degree (which implies that those numbers aren't very precise at all). By the way, if anyone is interested in the Bennett sight-reduction table for altitude, which tabulates three numbers for each minute of arc between 0 deg and 90 deg, those three quantities appear to be, in order- -13,030 log cos (lat or dec) -13,030 log haversine LHA 200,000 haversine (lat ~ dec) where the haversine of an angle is (1-cos (angle))/2 ==================== Peter has provided the following information- >George Bennett has made a preliminary examination of the probability of >error using these azimuth tables. The following is taken from that work. >Examples of Rounding-off errors in Sight Reduction Tables and Azimuth >Tables Now that's interesting! I don't remember seeing that work elsewhere. Was it posted to Nav-L? No warning, or analysis, about azimuth errors seems to exist on the Bennett website. I would like to read the full text rather than extracts. Where can I do so, Peter? I would like to understand Bennett's analysis methods. Bennett adds- >I have added a footnote in the new edition which warns the user to take >care when observations have been made near the PV and substantial >rounding off must be made. When in doubt, interpolate. Good. I raised the subject in the first place because no such warning had been given in my copy. And in that edition, the user was clearly told that interpolation was unnecessary. Peter Fogg commented- >The highly selective >example showing a 15 degree error is too remote to be of practical concern. It was deliberately chosen to be an extreme example (but not a unique one) to show the maximum errors that could occur. There's a wide range of combinations of LHA, dec, and alt which can give rise to errors in azimuth up to, but less than, that 15 degree limit. George. ================================================================ contact George Huxtable by email at george---.u-net.com, by phone at 01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK. ================================================================