# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: formula for refraction**

**From:**Frank Reed CT

**Date:**2007 Mar 21, 02:20 EDT

"Meeus's formula is certainly approximate (and he says so)

but it does not match the almanac table with the precision

Meeus claims.

Chauvenet has complete theory but no useful simple formula.

My question is practical: I want to incorporate automatic refraction

computation in my spreadsheet for star distances."

but it does not match the almanac table with the precision

Meeus claims.

Chauvenet has complete theory but no useful simple formula.

My question is practical: I want to incorporate automatic refraction

computation in my spreadsheet for star distances."

The refraction tables in the almanacs are derived from numerical
integrations which are confirmed by detailed observations at observatories. The
resulting table can be matched to numerical series of various flavors. Depending
on the spreadsheet you're using, you could just input the whole table giving the
refraction for each (exact) degree of altitude, and interpolate (use a "VLOOKUP"
function --see detailed formula in PS below).

The formulas given in various sources on the pattern of

Ro = 0d.0167/tan(H + 7.32/(H + 4.32))

are simply numerical "fittings". They're not "derived" from underlying theory. In my code, I use two different equations, one for altitudes above 15 degrees, one for lower altitudes. At higher altitudes, R = 57"/tan(H) is excellent. If you adjust the parameters in the other formula (above), you can get better results at lower altitudes. From the perspective of accuracy and technical quality, this is NOT a better solution than inputting the whole table, but it certainly involves less typing. :-)

are simply numerical "fittings". They're not "derived" from underlying theory. In my code, I use two different equations, one for altitudes above 15 degrees, one for lower altitudes. At higher altitudes, R = 57"/tan(H) is excellent. If you adjust the parameters in the other formula (above), you can get better results at lower altitudes. From the perspective of accuracy and technical quality, this is NOT a better solution than inputting the whole table, but it certainly involves less typing. :-)

PS: Some spreadsheet details... In cells G10 through G100, fill in
integral degrees of altitude 0, 1, 2, ...90. Then in H10 through H100, enter the
corresponding refraction in minutes of arc. For altitudes above 15 degrees, you
could simply enter the standard formula, rather than typing them in. For
lower altitudes, take the values from the almanac, and enter them
manually. Note that this block of values can be placed "way out in right
field" on your spreadsheet so that it doesn't clutter up your calculating area.
Next suppose I have an altitude in degrees in cell B2 for which I want an exact
value of the refraction. This would typically be a non-integral altitude, so we
need a formula to grab bracketing values from the refraction table (in G10:H100)
and interpolate between them. And here's the formula:

=VLOOKUP(INT(B2),G10:H100,1)*(1-(B2-INT(B2)))+VLOOKUP(INT(B2)+1,G10:H100,1)*(B2-INT(B2))

Naturally, it's a linear interpolation, which is more than adequate when
the altitude difference is 1 degree. Note that this formula would have to be
modified if you decide to use a separation different from 1 degree or a variable
separation, as in the Nautical Almanac. Enjoy!

AOL now offers free email to everyone. Find out more about what's free from AOL at

**AOL.com**.

--~--~---------~--~----~------------~-------~--~----~

To post to this group, send email to NavList@fer3.com

To unsubscribe, send email to NavList-unsubscribe@fer3.com

-~----------~----~----~----~------~----~------~--~---