# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Sight Reduction Formula**

**From:**George Huxtable

**Date:**2003 Oct 19, 14:50 +0100

Ralph Clampitt asked- >I am new to the list and have just recently started using a programmable >calculator for sight reductions. I have found several useful formulae on >this list and in other references, however the one shown in the Nautical >Almanac for azimuth puzzles me. >Can anyone explain the formula for azimuth from the Almanac? > >Summarizing from page 279 - 2002 Nautical Almanac > >6. The calculated altitude and azimuth. >Step 1. Calculate the local hour angle. > LHA = GHA + Long. > >Step 2. Calculate S, C and the altitude Hc from > S = sin Dec > C = cos Dec cos LHA >Sin Hc = (S sin LAT + C cos LAT) > >(This appears to be analogous to the fundamental cosine formula of >spherical trigonometry and to be the classic formula for computing >altitude. Most references however seem to use [+ or abs. difference]) >i.e. sin Hc = sin LAT * sin Dec [+ or absolute diff.] cos LAT * cos Dec * >cos LHA. > >The trouble, for me, starts in using the formula for azimuth from the >almanac, which seems to be quite different from those most commonly shown >in other references, such as one that seems fairly standard > > Cos Z = (sin Dec [+ or abs. diff.] sin Hc) / (cos Hc * cos Lat) > >The Almanac formula for azimuth show in Step 3 is as follows; > >Step 3. Calculate X and A from > X= (S cos Lat - C sin LAT)/cos Hc > Cos A = X > >Determine the azimuth Z >If HA > 180 degrees than Z= A >Otherwise Z = 360 degrees - A > >The formula than seems to be >Cos A = [(sin Dec * cos LAT) - (cos Dec *cos LHA * sin LAT)]/cos Hc > >As I said, I have not seen this formula anywhere else and also I have had >trouble with it when Dec and Lat are opposite signs. Please understand >that I am not much of a mathimatician. > >Ralph Clampitt ========================== Response from George- Welcome to the list, Ralph. Sorry, this isn't an answer to Ralph's specific question about the azimuth formula in the Nautical Almanac. It's a suggestion of a better way to do the job. One problem about getting the Az from its cos is that this becomes inaccurate for azimuths near to due North and South. Getting az from its sine is worse still, in that this is inaccurate near East-West directions, but also ambiguous about East and about West, and the ambiguity is hard to resolve. An additional difficulty with both methods above is that you need to calculate altitude first, whether or not the altitude was needed for the problem. Why not calculate azimuth directly from its tan, instead? use Az = arctan ((-Sin HA) / (Cos Lat * Tan Dec - Sin Lat * Cos HA)) Lat and dec are in the range -90 (south) through zero (equator) to +90 (north pole). HA increases westwards, 0 to 360. Az increases clockwise from North, 0 to 360. (It may, on some machines, be worthwhile to trap for a zero arising in the denominator.) With this formula, there are no regions of low precision, as there would be if calculating az from its sin or cos. And you can see that it requires only HA, lat, and dec, not a calculated altitude. However, the ordinary arc-tan function still has two possible results, so it's not yet quite clear of ambiguity. If you just have the standard arctan function available, without any facilities for rectangular-polar conversion, then a rule for unambiguously obtaining az is as follows- if tan az was negative, add 180 degrees to az. if LHA was less than 180 degrees, add (another) 180 degrees to az. Then if az exceeds 360 degrees, subtract 360. However, many calculators and computers make things easier by providing rectangular to polar conversion functions. These examine the signs of numerator and denominator separately. Many Casio calculators provide the POL (polar) function, with two inputs, and when you apply X = POL (( Cos Lat * Tan Dec - Sin Lat * Cos HA) , (- Sin HA)) then what appears in the Y variable is a quantity between -180 and +180, unambiguously defining the azimuth. (Negative azimuths can have 360 added to put them into familiar form.) Some computers provide a similar function which is often labelled ATAN2, to distinguish it from the normal ATAN inverse tangent. The signs of the two input terms must be carefully chosen, however (as they were for the POL example above). Rectangular to polar conversions are conventionally intended to produce an angle which increases anticlockwise from the x-axis, whereas we require an angle that increases clockwise from the y axis, so a bit of thoughtful tinkering may be called for. Using RP notation on my old HP21, still going strong after 30 years, the technique was to- Enter (-sin HA) work out (cos HA sin lat - cos lat tan dec) convert to (R, theta) look at theta and if negative add 360. When using these methods you have to be careful with the signs. For example, in the formula for Az, using POL, above, DON'T try to cancel out the negative sign in the second input term by reversing the order of the subtraction in the first term! The solution for Az, from its tan, won't be found in many navigational textbooks, and I think the reason is partly historical. Using this method, the terms involved can be positive or negative. Old navigators would avoid calculating with anything going negative if they possibly could, largely because everything relied on logs, which could only handle positive quantities. In general, that's why traditional formulae would use names, such as North or South, rather than signs, and (sum-or-difference) expressions rather than subtraction, to avoid handling negatives. Nowadays, we no longer use logs, and are more confident in manipulating negative quantities. This explains the difference Ralph notes, between the mathematically straightforward expression for altitude in the modern almanac, and the "classic formula" he has found in texts. The derivation of az from its tan is given in Jean Meeus' essential "Astronomical Algorithms", as eq. 13.5 You have to be a bit careful with Meeus, however, because as an astronomer he measures his azimuths clockwise from the South, adding notes to that effect on pages 91 and 92. George Huxtable. ================================================================ contact George Huxtable by email at george@huxtable.u-net.com, by phone at 01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK. ================================================================