# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Sight Reduction Formula**

**From:**Ralph Clampitt

**Date:**2003 Oct 20, 13:01 -0800

George, Thanks for the suggestion. I have seen other formula calculating from the tangent such as Az = arctan((sin LHA)/(sin Lat * cos LHA) - (tan Dec * cos Lat) but have not tried them. Our cruising area is mostly around 58 - 60 degrees north in the Gulf of Alaska, so hadn't worried much about inaccuracies in either near North-South or near East-West directions. Should we? Will check out your proposed use of tan fromula. Ralph Clampitt -----Original Message----- From: George Huxtable [mailto:george{at}HUXTABLE.U-NET.COM] Sent: Sunday, October 19, 2003 3:37 PM To: NAVIGATION-L{at}LISTSERV.WEBKAHUNA.COM Subject: Re: Sight Reduction Formula Response from George- Welcome to the list, Ralph. Sorry, this isn't an answer to Ralph's specific question about the azimuth formula in the Nautical Almanac. It's a suggestion of a better way to do the job. One problem about getting the Az from its cos is that this becomes inaccurate for azimuths near to due North and South. Getting az from its sine is worse still, in that this is inaccurate near East-West directions, but also ambiguous about East and about West, and the ambiguity is hard to resolve. An additional difficulty with both methods above is that you need to calculate altitude first, whether or not the altitude was needed for the problem. Why not calculate azimuth directly from its tan, instead? use Az = arctan ((-Sin HA) / (Cos Lat * Tan Dec - Sin Lat * Cos HA)) Lat and dec are in the range -90 (south) through zero (equator) to +90 (north pole). HA increases westwards, 0 to 360. Az increases clockwise from North, 0 to 360. (It may, on some machines, be worthwhile to trap for a zero arising in the denominator.) With this formula, there are no regions of low precision, as there would be if calculating az from its sin or cos. And you can see that it requires only HA, lat, and dec, not a calculated altitude. However, the ordinary arc-tan function still has two possible results, so it's not yet quite clear of ambiguity. If you just have the standard arctan function available, without any facilities for rectangular-polar conversion, then a rule for unambiguously obtaining az is as follows- if tan az was negative, add 180 degrees to az. if LHA was less than 180 degrees, add (another) 180 degrees to az. Then if az exceeds 360 degrees, subtract 360. However, many calculators and computers make things easier by providing rectangular to polar conversion functions. These examine the signs of numerator and denominator separately. Many Casio calculators provide the POL (polar) function, with two inputs, and when you apply X = POL (( Cos Lat * Tan Dec - Sin Lat * Cos HA) , (- Sin HA)) then what appears in the Y variable is a quantity between -180 and +180, unambiguously defining the azimuth. (Negative azimuths can have 360 added to put them into familiar form.) Some computers provide a similar function which is often labelled ATAN2, to distinguish it from the normal ATAN inverse tangent. The signs of the two input terms must be carefully chosen, however (as they were for the POL example above). Rectangular to polar conversions are conventionally intended to produce an angle which increases anticlockwise from the x-axis, whereas we require an angle that increases clockwise from the y axis, so a bit of thoughtful tinkering may be called for. Using RP notation on my old HP21, still going strong after 30 years, the technique was to- Enter (-sin HA) work out (cos HA sin lat - cos lat tan dec) convert to (R, theta) look at theta and if negative add 360. When using these methods you have to be careful with the signs. For example, in the formula for Az, using POL, above, DON'T try to cancel out the negative sign in the second input term by reversing the order of the subtraction in the first term! The solution for Az, from its tan, won't be found in many navigational textbooks, and I think the reason is partly historical. Using this method, the terms involved can be positive or negative. Old navigators would avoid calculating with anything going negative if they possibly could, largely because everything relied on logs, which could only handle positive quantities. In general, that's why traditional formulae would use names, such as North or South, rather than signs, and (sum-or-difference) expressions rather than subtraction, to avoid handling negatives. Nowadays, we no longer use logs, and are more confident in manipulating negative quantities. This explains the difference Ralph notes, between the mathematically straightforward expression for altitude in the modern almanac, and the "classic formula" he has found in texts. The derivation of az from its tan is given in Jean Meeus' essential "Astronomical Algorithms", as eq. 13.5 You have to be a bit careful with Meeus, however, because as an astronomer he measures his azimuths clockwise from the South, adding notes to that effect on pages 91 and 92. George Huxtable. ================================================================ contact George Huxtable by email at , by phone at 01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK.