NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
From: Frank Reed
Date: 2021 May 21, 10:11 -0700
Ah yes, the effect of altitude on altitudes... That's an interesting problem with the English language! How does altitude above sea level affect the angular altitude of a celestial body?
For angular altitudes, alt, above 15°, the refraction correction for a sextant sight can be calculated with high accuracy from:
r0 = 0.97' / tan(alt)
which applies at standard temperature T0 = 50°F/10°C and pressure P0 = 29.80 in Hg/ 1010mb. At higher pressures, the refraction is increased in direct proportion so you multiply that r0 value by P/P0, which is easy. It makes sense, too. Air under higher pressure is thicker, higher-density air, and greater density means more refraction. The refraction also increases in inverse proportion to the absolute temperature. Hotter air is thinner air. So first we convert the current and standard temperatures to an absolute scale and then we divide by their ratio.
This is not as compllicated as it sounds. Work out the baseline refraction for some angular altitude. For example, suppose your angular altitude is 28.5°. Then 0.97'/tan(28.5°) is just about 1.8'. Now suppose your barometer or the internet tells you that air pressure is 1020mb (staying in SI units from here). Compare that with the standard pressure and you will find that this is 1% above standard pressure. So the refraction is increased by 1%. That shifts the refraction to 1.818' and rounded to two digits, the result is unchanged: 1.8'. Suppose also that your air temperature is 38°C (it's hot). Converted to kelvins by adding 273, that's 311K while the standard temp is 283K. The ratio of those two absolute temperatures is almost exactly 1.10 which implies that the air is 10% thinner than at standard temp. The refraction changes in inverse proportion. It is reduced by 10%. And 10% of 1.8' is just about 0.2' so finally the refraction is 1.6'. That's easy, right? And notice that it's really not much different, and it won't be much of a change unless you have relatively large baseline refraction values, which only happens at lower angular altitudes.
What should we do if we climb a mountain or fly in an aircraft? One option is to measure the local pressure and temperature and altitude and then proceed exactly as above. That's fine, but there are other approaches. If you don't have local measures of T and P, you can use the normal decrease in atmospheric density with altitude which has an "e-folding" height of about 9.5km, and that's equivalent to a "halving" height of 6.6km, meaning that the density of air, on average, decreases by a factor of two for every 6.6km that you climb above sea level. In more formal mathematical terms, you can take the sea level refraction, r0, and multiply by exp(-h/9.5km) or by 2^(-h/6.6km).
Suppose we have the same initial angular altitude as above, implying that r0 is 1.8'. Suppose also that we are in an airplane flying at 13.2km (that's in the lower stratosphere). That altitude is equal to two "halving" altitudes so air density is reduced by a factor of four. Then the refraction is 0.45'. Note that there's no separate correction needed here for the low outside air temperature. That's already included.
For a case that's more relevant to a land explorer, suppose your altitude is 1km. And just to change things up for this example, let's suppose that the angular altitude is 46.3°. The sea level refraction would be 0.97'/tan(46.3°) which is 0.93'. The factor for altitude above sea level is:
2(-1km/6.6km)
or 2^(-1/6.6) which is about 0.90 implying that the refraction is 0.9 · 0.93' or 0.84'. If you experiment with cases, you'll find that the decrease in air density for low altitudes above sea level is almost exactly 1% per 100m for altitudes below roughly 2500m. And that 1% change per 100m also applies directly to the refraction. And that's telling us that "it ain't much". Imagine looking at a star quite low in the sky with a standard refraction of 10.0'. If you were to climb 500m above sea level, the refraction would drop by 1% per 100m or just about 5%, and that would lower the refraction by 5% to 9.5'. There are not many cases in celestial navigation where this would be a significant concern so this case qualifies as a sort of lower limit.
Frank Reed
