# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Refraction for beginners**

**From:**Frank Reed CT

**Date:**2004 Jul 9, 20:47 EDT

Bill you wrote:

"His rule of thumb was plus/minus 1" mercury per 1000 ft change."

Right. That's a popular rough rule of thumb. Naturally, it can't go this way forever or the atmosphere would come to a very abrupt end at 29,920 feet! Below 5000 to 10,000 feet the "one inch per thousand feet" rule is not bad, but as you climb higher the rate of change reduces (there's less atmosphere left up there). For example, at 40,000 feet, you would have to climb about 3800 feet to see the pressure drop by 1 more inch.

It's interesting (well, as interesting as this sort of stuff can get <g>) that the percentage change (as opposed to the additive change) is nearly the same at all altitudes. If you climb 1600 feet into the atmosphere from sea level the pressure drops just about 5%. If you write this new pressure down and climb another 1600 feet, the pressure drops by just about 5% from that new pressure you wrote down. Even at 50,000 feet, if you climb 1600 feet, you will find that the pressure at 51,600 is lower by nearly 5% from the pressure you left behind at 50,000. This happens because, from a mathematical point of view, the density of the atmosphere drops exponentially with altitude roughly according to the formula

(density at height h) = (density at sea level) * exp( - height / 32000feet).

And you wrote:

"So it is not just what happens at the "boundary" of outer space and Earth's atmosphere, but also how much air (distance) there is between the boundary and observer, and how dense it is."

Exactly. The air gets thinner as you climb so the refraction is smaller. And of course there really is no "boundary" where the atmosphere suddenly ends. It just keeps getting thinner and thinner. This issue came up recently when Burt Rutan's private space plane made it "into space" for a few seconds. It's only "space" as defined by aeronautical record books. No satellites orbit at that altitude, but there's VERY little air up there.

And wrote of the Great Lakes (some 600 feet above sea level):

"Since we are only talking about .6" of mercury, that would keep me within

the limits the temp/pressure table."

Right and that's only a small correction. For normal celestial navigation, it's really not worth worrying about, but if you ever find yourself practicing cel nav sights when you're in the mountains, then it can be an important issue.

And wrote:

"If I were in Denver and used local pressure (maybe 25" mercury), I would be

outside the table's parameters anyway."

Yeah, kindof a problem, huh? This actually was a practical issue back when high-altitude airplanes were navigated by sextant, especially in the 1940s and 50s. Instead of extending the barometric pressure correction, air navigators had a separated "altitude in feet" correction that they applied to their sights. Today, if you want a "height above sea level correction" you could generate one for your location using the refraction for stars in the Nautical Almanac. You would multiply the values in that table by a factor which depends exponentially on altitude above sea level. For Denver the factor would be close to 0.16. So you would take the whole star altitude correction table and multiply the values by 0.16. For example, at 10 degrees altitude, the correction is -5.3 minutes. Multiplied by 16% gives 0.8 minutes. This is an additive correction, opposite in sign to the original correction. At 60 degrees above the horizon, the star refraction correction is -0.6 minutes, so the "height above sea level correction" would be 0.1 minutes. For any sight that you do after constructing this table, you would add in these values. So if you shoot an altitude of the Moon's Lower Limb tomorrow in Denver, and you find that it is 60 degrees, the Nautical Almanac gives a correction of +15.4 minutes. You would add 0.1 minutes to this altitude. Clearly a minor correction. On the other hand, if you later found the Moon's LL altitude to be 10 degrees, the correction from the almanac is +10.7 minutes and you would have to add 0.8 minutes to this --still small, but definitely worth some attention.

I should emphasize that this whole business is mostly of theoretical interest. Most navigators never worry about any of this "altitude above sea level" nonsense since celestial navigation almost always occurs at or close to sea level. It is ONLY an issue if you do practice sights from locations well above sea level or you happen to be using a sextant aboard a high-altitude jet (very unlikely these days).

Frank R

[ ] Mystic, Connecticut

[X] Chicago, Illinois

"His rule of thumb was plus/minus 1" mercury per 1000 ft change."

Right. That's a popular rough rule of thumb. Naturally, it can't go this way forever or the atmosphere would come to a very abrupt end at 29,920 feet! Below 5000 to 10,000 feet the "one inch per thousand feet" rule is not bad, but as you climb higher the rate of change reduces (there's less atmosphere left up there). For example, at 40,000 feet, you would have to climb about 3800 feet to see the pressure drop by 1 more inch.

It's interesting (well, as interesting as this sort of stuff can get <g>) that the percentage change (as opposed to the additive change) is nearly the same at all altitudes. If you climb 1600 feet into the atmosphere from sea level the pressure drops just about 5%. If you write this new pressure down and climb another 1600 feet, the pressure drops by just about 5% from that new pressure you wrote down. Even at 50,000 feet, if you climb 1600 feet, you will find that the pressure at 51,600 is lower by nearly 5% from the pressure you left behind at 50,000. This happens because, from a mathematical point of view, the density of the atmosphere drops exponentially with altitude roughly according to the formula

(density at height h) = (density at sea level) * exp( - height / 32000feet).

And you wrote:

"So it is not just what happens at the "boundary" of outer space and Earth's atmosphere, but also how much air (distance) there is between the boundary and observer, and how dense it is."

Exactly. The air gets thinner as you climb so the refraction is smaller. And of course there really is no "boundary" where the atmosphere suddenly ends. It just keeps getting thinner and thinner. This issue came up recently when Burt Rutan's private space plane made it "into space" for a few seconds. It's only "space" as defined by aeronautical record books. No satellites orbit at that altitude, but there's VERY little air up there.

And wrote of the Great Lakes (some 600 feet above sea level):

"Since we are only talking about .6" of mercury, that would keep me within

the limits the temp/pressure table."

Right and that's only a small correction. For normal celestial navigation, it's really not worth worrying about, but if you ever find yourself practicing cel nav sights when you're in the mountains, then it can be an important issue.

And wrote:

"If I were in Denver and used local pressure (maybe 25" mercury), I would be

outside the table's parameters anyway."

Yeah, kindof a problem, huh? This actually was a practical issue back when high-altitude airplanes were navigated by sextant, especially in the 1940s and 50s. Instead of extending the barometric pressure correction, air navigators had a separated "altitude in feet" correction that they applied to their sights. Today, if you want a "height above sea level correction" you could generate one for your location using the refraction for stars in the Nautical Almanac. You would multiply the values in that table by a factor which depends exponentially on altitude above sea level. For Denver the factor would be close to 0.16. So you would take the whole star altitude correction table and multiply the values by 0.16. For example, at 10 degrees altitude, the correction is -5.3 minutes. Multiplied by 16% gives 0.8 minutes. This is an additive correction, opposite in sign to the original correction. At 60 degrees above the horizon, the star refraction correction is -0.6 minutes, so the "height above sea level correction" would be 0.1 minutes. For any sight that you do after constructing this table, you would add in these values. So if you shoot an altitude of the Moon's Lower Limb tomorrow in Denver, and you find that it is 60 degrees, the Nautical Almanac gives a correction of +15.4 minutes. You would add 0.1 minutes to this altitude. Clearly a minor correction. On the other hand, if you later found the Moon's LL altitude to be 10 degrees, the correction from the almanac is +10.7 minutes and you would have to add 0.8 minutes to this --still small, but definitely worth some attention.

I should emphasize that this whole business is mostly of theoretical interest. Most navigators never worry about any of this "altitude above sea level" nonsense since celestial navigation almost always occurs at or close to sea level. It is ONLY an issue if you do practice sights from locations well above sea level or you happen to be using a sextant aboard a high-altitude jet (very unlikely these days).

Frank R

[ ] Mystic, Connecticut

[X] Chicago, Illinois