# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Refraction formula comparison
From: Paul Hirose
Date: 2000 Oct 01, 2:37 PM

```The Explanatory Supplement to the Astronomical Almanac gives this
low-precision formula for refraction below 15 degrees:

R = (P / (T + 273)) * (.1594 + .0196 * H + .00002 * H^2) / (1 + .505
* H + .0845 * H^2)

where H = apparent altitude, P = air pressure in millibars, and T =
degrees Celsius. Result is in degrees.

The Explanatory Supplement also gives first-order approximations to
correct celestial coordinates for refraction. I will not give them
here, but they are simple formulas. You could use them to determine
the refraction-corrected separation angle between stars.

That was some sharp thinking by George Huxtable, to spot the problem
lurking in the Bennett refraction formula. I'd expect the exact
location of the singularity to depend on your machine. With my HP-15C
calculator, it's at 89.92249993. Any change in the last decimal place
avoids it.

George is quite correct that a programmer should not shrug off a
potential bomb like this. I once crashed an astronomy program by
innocently entering exactly 90 degrees for a parameter.

Here is a comparison of the refraction formulas:

H  Bennett    Supp.     Keys   Smart

0     34.5     34.1     35.0      -
1     24.3     24.1     25.4   151.1
2     18.2     18.1     18.8     2.0
5      9.9      9.8      9.9     9.5
10      5.4      5.3      5.3     5.3
12      4.5      4.4      4.5     4.5
15      3.6      3.6      3.6     3.6
30      1.7      1.8      1.7     1.7

Except for formula no. 2, all are equal at higher altitudes.

The Explanatory Supplement formula is only supposed to be used below
15 degrees, and the Smart formula from 15 up, but both are good for
navigation outside those ranges. I included 12 degrees because that's
where the maximum error of the Bennett formula is supposed to occur.
```
Browse Files

Drop Files

### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

### Email Settings

 Posting Code:

### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)