# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Sight Reduction Formula Question**

**From:**George Huxtable

**Date:**2005 Jan 7, 23:30 +0000

Fried Squash wrote- >Thanks Fred, you are absolutely correct, Turner does >indeed explain that the short one can yield 180 deg >ambiguity. I should have kept reading. ===================== From George. No, not just a 180 deg ambiguity. It's far more awkward than that. A 180 degree ambiguity would be rather easy to allow for. If you derive Z from its cosine, then when cos Z = +.98 (for example), then Z = 11.5 degrees. Or else -11.5 degrees (same as 348.5 deg). It's not too hard to sort out which, between those two, because if you sketch it out, it's obvious, if the body is passing to your North, that if its still to your East, then it has to be in the NE sector, so Z must be 11.5. So it depends on your lat, the dec of the body, and the sign of its its LHA, all of which are known. In the convention I use, North is positive, and West is positive, so hour angles are always increasing, and Az is measured clockwise from North (which differs from the convention used by many astronomers). For the formula that derives Z from its cosine, the ambiguity problem is easy to resolve, by inspection. However, another problem remains: that near azimuths of O deg (North) and 180 deg (South), the cosine changes hardly at all for quite large changes of azimuth. So in order to obtain the azimuth with any accuracy when near due North or due South, you need to know its cosine VERY precisely indeed. That implies that the altitude H, which was calculated at an earlier stage, also needs to have been obtained to a high accuracy. Otherwise you might find that cos Z might turn out to be greater than 1, which would be embarrassing. Fortunately Z is seldom required to great accuracy. Now consider the other formula, deriving Z from its sine. If, in this case, that formula has given us sin Z = +0.98, what do we do then? Well, one solution is that Z = 88.5 deg, which might well be the right answer. On the other hand, sin 101.5 deg is also +0.98, so that's an equally valid answer. It could be either, then. How do we discover which is the one we want? In this case, there's no simple way, that I know of, to resolve the ambiguity by using inspection and logic to determine whether the wanted result is just North of East or just South of East. You will find the advice, in some textbooks, "just take a compass bearing on the body, and it should be easy to distinguish which one is right". Well, perhaps that's reasonable, in that widely-spread example, but say the two possible angles were closer, say 88 deg and 92 deg. Would you be sure which was the right one then? And anyway, if you're prepared to accept a compass-bearing for azimuth, why are you bothering to calculate it? No, deriving azimuth from its sine is the worst possible option. In addition, it has the same problem of inaccuracy as the cos Z method, but this time for azimuths that are near East and West. There's a third option, that for some reason doesn't find its way into many textbooks. Get the azimuth from its tan! This formula is- Tan Z = sin (hour-angle) / (cos (hour-angle) sin lat - cos lat tan dec) and the rules for putting Z into the right quadrant, 0 to 360, clockwise fron North, are- If tan Z was negative, add 180 deg to Z. If hour-angle was less than 180 deg, add another 180 deg to Z. You can see from this that the calculated altitude doesn't even enter into the formula above, so there's no longer a requirement to calculate altitude at all, if only an azimuth was going to be needed. With a programmable calculator or computer which can derive polar coordinates using the "POL" or "ATAN2" function, it's even simpler, as that function is designed to put the result into the correct quadrant automatically. For example, in Casio Basic, use X = POL ((Tan dec cos lat - cos (hour-angle) sin lat), - sin (hour-angle)) and then the azimuth will appear in variable Y, as an angle measured clockwise from North between -180 and + 180 deg. You can put this into 0 to 360 notation, if you need to, by simply adding 360 deg if it's negative. But don't try simplifying out that expression above for X by cancelling any minus-signs, or you will spoil it. Using the tan Z expression, then tan Z is changing quickly with azimuth at all angles, so there are no "dead-spots" with low accuracy, as exist in the other methods. I don't understand why it's not used more widely. George. >--- Fred Hebardwrote: >> I believe Turner gives the reason for the longer >> formula and also shows >> the shorter one, saying the shorter one can be >> inaccurate in some cases. >> Fred >> >> On Jan 7, 2005, at 10:24 AM, Fried Squash wrote: >> > In _Celestial for the Cruising Navigator_ by Merle >> > Turner, the navigational triangle azimuth formula >> is >> > given as: >> > >> > cos Z = (sin d - (sin L * sin H ))/(cos L * cos H) >> > >> > but Dutton's Navigation and Piloting gives the >> much >> > simpler: >> > >> > sin Z = ( cos d * sin t ) / cos H >> > >> > They both seem to work. But there must be some >> reason >> > that the longer one is better, right? Works in >> more ================================================================ contact George Huxtable by email at george@huxtable.u-net.com, by phone at 01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK. ================================================================