# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Sight Reduction Formula Question**

**From:**Bill B

**Date:**2005 Jan 7, 23:10 -0500

George A lot for a liberal arts major to follow. My question. Susan Howell states (for the longer of the two formulas Fried gave) "if sin LHA is less than 0, Zn = Z, if sin LHA is equal to or greater than 0, Zn = 360 - Z." Do you perceive this as a reasonable litmus test, or is it flawed in your learned opinion? More probable, have I entirely missed the point? Thanks Bill > From George. > > No, not just a 180 deg ambiguity. It's far more awkward than that. A 180 > degree ambiguity would be rather easy to allow for. > > If you derive Z from its cosine, then when cos Z = +.98 (for example), then > Z = 11.5 degrees. Or else -11.5 degrees (same as 348.5 deg). It's not too > hard to sort out which, between those two, because if you sketch it out, > it's obvious, if the body is passing to your North, that if its still to > your East, then it has to be in the NE sector, so Z must be 11.5. So it > depends on your lat, the dec of the body, and the sign of its its LHA, all > of which are known. > > In the convention I use, North is positive, and West is positive, so hour > angles are always increasing, and Az is measured clockwise from North > (which differs from the convention used by many astronomers). > > For the formula that derives Z from its cosine, the ambiguity problem is > easy to resolve, by inspection. However, another problem remains: that near > azimuths of O deg (North) and 180 deg (South), the cosine changes hardly at > all for quite large changes of azimuth. So in order to obtain the azimuth > with any accuracy when near due North or due South, you need to know its > cosine VERY precisely indeed. That implies that the altitude H, which was > calculated at an earlier stage, also needs to have been obtained to a high > accuracy. Otherwise you might find that cos Z might turn out to be greater > than 1, which would be embarrassing. Fortunately Z is seldom required to > great accuracy. > > Now consider the other formula, deriving Z from its sine. If, in this case, > that formula has given us sin Z = +0.98, what do we do then? Well, one > solution is that Z = 88.5 deg, which might well be the right answer. On the > other hand, sin 101.5 deg is also +0.98, so that's an equally valid answer. > It could be either, then. How do we discover which is the one we want? In > this case, there's no simple way, that I know of, to resolve the ambiguity > by using inspection and logic to determine whether the wanted result is > just North of East or just South of East. > > You will find the advice, in some textbooks, "just take a compass bearing > on the body, and it should be easy to distinguish which one is right". > Well, perhaps that's reasonable, in that widely-spread example, but say the > two possible angles were closer, say 88 deg and 92 deg. Would you be sure > which was the right one then? And anyway, if you're prepared to accept a > compass-bearing for azimuth, why are you bothering to calculate it? > > No, deriving azimuth from its sine is the worst possible option. In > addition, it has the same problem of inaccuracy as the cos Z method, but > this time for azimuths that are near East and West. > > There's a third option, that for some reason doesn't find its way into many > textbooks. Get the azimuth from its tan! > > This formula is- > > Tan Z = sin (hour-angle) / (cos (hour-angle) sin lat - cos lat tan dec) > > and the rules for putting Z into the right quadrant, 0 to 360, clockwise > fron North, are- > > If tan Z was negative, add 180 deg to Z. > If hour-angle was less than 180 deg, add another 180 deg to Z. > > You can see from this that the calculated altitude doesn't even enter into > the formula above, so there's no longer a requirement to calculate altitude > at all, if only an azimuth was going to be needed. > > With a programmable calculator or computer which can derive polar > coordinates using the "POL" or "ATAN2" function, it's even simpler, as that > function is designed to put the result into the correct quadrant > automatically. > > For example, in Casio Basic, > use X = POL ((Tan dec cos lat - cos (hour-angle) sin lat), - sin (hour-angle)) > > and then the azimuth will appear in variable Y, as an angle measured > clockwise from North between -180 and + 180 deg. You can put this into 0 to > 360 notation, if you need to, by simply adding 360 deg if it's negative. > > But don't try simplifying out that expression above for X by cancelling any > minus-signs, or you will spoil it. > > Using the tan Z expression, then tan Z is changing quickly with azimuth at > all angles, so there are no "dead-spots" with low accuracy, as exist in the > other methods. I don't understand why it's not used more widely. > > George. > > >> --- Fred Hebardwrote: >>> I believe Turner gives the reason for the longer >>> formula and also shows >>> the shorter one, saying the shorter one can be >>> inaccurate in some cases. >>> Fred >>> >>> On Jan 7, 2005, at 10:24 AM, Fried Squash wrote: >>>> In _Celestial for the Cruising Navigator_ by Merle >>>> Turner, the navigational triangle azimuth formula >>> is >>>> given as: >>>> >>>> cos Z = (sin d - (sin L * sin H ))/(cos L * cos H) >>>> >>>> but Dutton's Navigation and Piloting gives the >>> much >>>> simpler: >>>> >>>> sin Z = ( cos d * sin t ) / cos H >>>> >>>> They both seem to work. But there must be some >>> reason >>>> that the longer one is better, right? Works in >>> more