# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: Sight Reduction Formula
From: George Huxtable
Date: 2003 Oct 19, 14:50 +0100

```Ralph Clampitt asked-

>I am new to the list and have just recently started using a programmable
>calculator for sight reductions.  I have found several useful formulae on
>this list and in other references, however the one shown in the Nautical
>Almanac for azimuth puzzles me.
>Can anyone explain the formula for azimuth from the Almanac?
>
>Summarizing from page 279 - 2002 Nautical Almanac
>
>6. The calculated altitude and azimuth.
>Step 1.  Calculate the local hour angle.
>        LHA = GHA + Long.
>
>Step 2.  Calculate S, C and the altitude Hc from
>        S = sin Dec
>        C = cos Dec cos LHA
>Sin Hc = (S sin LAT + C cos LAT)
>
>(This appears to be analogous to the fundamental cosine formula of
>spherical trigonometry and to be the classic formula for computing
>altitude.   Most references however seem to use [+ or abs. difference])
>i.e. sin Hc = sin LAT * sin Dec [+ or absolute diff.] cos LAT * cos Dec *
>cos LHA.
>
>The trouble, for me, starts in using the formula for azimuth from the
>almanac, which seems to be quite different from those most commonly shown
>in other references, such as one that seems fairly standard
>
>        Cos Z = (sin Dec  [+ or abs. diff.] sin Hc) / (cos Hc * cos Lat)
>
>The Almanac formula for azimuth show in Step 3 is as follows;
>
>Step 3. Calculate X and A from
>        X= (S cos Lat - C sin LAT)/cos Hc
>        Cos A = X
>
>Determine the azimuth Z
>If HA > 180 degrees than Z= A
>Otherwise Z = 360 degrees - A
>
>The formula than seems to be
>Cos A = [(sin Dec * cos LAT) - (cos Dec *cos LHA * sin LAT)]/cos Hc
>
>As I said, I have not seen this formula anywhere else and also I have had
>trouble with it when Dec and Lat are opposite signs. Please understand
>that I am not much of a mathimatician.
>
>Ralph Clampitt

==========================

Response from George-

Welcome to the list, Ralph.

Sorry, this isn't an answer to Ralph's specific question about the azimuth
formula in the Nautical Almanac. It's a suggestion of a better way to do
the job.

One problem about getting the Az from its cos is that this becomes
inaccurate for azimuths near to due North and South. Getting az from its
sine is worse still, in that this is inaccurate near East-West directions,
but also ambiguous about East and about West, and the ambiguity is hard to
resolve.

An additional difficulty with both methods above is that you need to
calculate altitude first, whether or not the altitude was needed for the
problem.

Why not calculate azimuth directly from its tan, instead?

use Az = arctan ((-Sin HA) / (Cos Lat * Tan Dec - Sin Lat * Cos HA))

Lat and dec are in the range -90 (south) through zero (equator) to +90
(north pole).
HA increases westwards, 0 to 360.
Az increases clockwise from North, 0 to 360.

(It may, on some machines, be worthwhile to trap for a zero arising in the
denominator.)

With this formula, there are no regions of low precision, as there would be
if calculating az from its sin or cos. And you can see that it requires
only HA, lat, and dec, not a calculated altitude.

However, the ordinary arc-tan function still has two possible results, so
it's not yet quite clear of ambiguity.

If you just have the standard arctan function available, without any
facilities for rectangular-polar conversion, then a rule for unambiguously
obtaining az is as follows-
if tan az was negative, add 180 degrees to az.
if LHA was less than 180 degrees, add (another) 180 degrees to az.
Then if az exceeds 360 degrees, subtract 360.

However, many calculators and computers make things easier by providing
rectangular to polar conversion functions. These examine the signs of
numerator and denominator separately.

Many Casio calculators provide the POL (polar) function, with two inputs,
and when you apply

X = POL (( Cos Lat * Tan Dec - Sin Lat * Cos HA) , (- Sin HA))

then what appears in the Y variable is a quantity between -180 and +180,
unambiguously defining the azimuth. (Negative azimuths can have 360 added
to put them into familiar form.)

Some computers provide a similar function which is often labelled ATAN2, to
distinguish it from the normal ATAN inverse tangent. The  signs of the two
input terms must be carefully chosen, however (as they were for the POL
example above). Rectangular to polar conversions are conventionally
intended to produce an angle which increases anticlockwise from the x-axis,
whereas we require an angle that increases clockwise from the y axis, so a
bit of thoughtful tinkering may be called for.

Using RP notation on my old HP21, still going strong after 30 years, the
technique was to-
Enter (-sin HA)
work out (cos HA sin lat - cos lat tan dec)
convert to (R, theta)
look at theta and if negative add 360.

When using these methods you have to be careful with the signs. For
example, in the formula for Az, using POL, above, DON'T try to cancel out
the negative sign in the second input term by reversing the order of the
subtraction in the first term!

The solution for Az, from its tan, won't be found in many navigational
textbooks, and I think the reason is partly historical. Using this method,
the terms involved can be positive or negative. Old navigators would avoid
calculating with anything going negative if they possibly could, largely
because everything relied on logs, which could only handle positive
quantities.

In general, that's why traditional formulae would use names, such as North
or South, rather than signs, and (sum-or-difference) expressions rather
than subtraction, to avoid handling negatives. Nowadays, we no longer use
logs, and are more confident in manipulating negative quantities. This
explains the difference Ralph notes, between the mathematically
straightforward expression for altitude in the modern almanac, and the
"classic formula" he has found in texts.

The derivation of az from its tan is given in Jean Meeus' essential
"Astronomical Algorithms", as eq. 13.5

You have to be a bit careful with Meeus, however, because as an astronomer
he measures his azimuths clockwise from the South, adding notes to that
effect on pages 91 and 92.

George Huxtable.

================================================================
contact George Huxtable by email at george@huxtable.u-net.com, by phone at
01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy
Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK.
================================================================

```
Browse Files

Drop Files

### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

### Email Settings

 Posting Code:

### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)