# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Refraction**

**From:**Marcel Tschudin

**Date:**2005 Aug 13, 17:00 +0300

George, YES, you are right and I was wrong! What I mentioned with ... >>To my oppinion this resulting refraction can in this case be corrected for >>air pressure and temperature as indicated in the textbooks. To my >>understanding it is the temperature and air pressure at the eye of the >>observer which corrects/adjusts the complete line of sight. ... is only valid within a limited range, the "astronomical range", of altitude and is therefore not applicable for low altitudes. Without looking at the physics behind it, I took this by mistake as a general valid rool, but this is - as you explained it - definitely not so. Before going further with an approximation formula for refraction values of negative altitudes, I would like to summarise of what I found so far. From what I can see, I will again have to come back on the subject of an approximation and on what you proposed. While you were afloat, I transferred the data of table 6 (only the range 0ft to 35'000ft) into an Excel sheet. I actually also tried to get in contact with the authors of this table, unfortunately all efforts failed so far. Pub. No. 249 is originally from NIMA. It seems that after the reorganisation - NIMA either became or was merged with NGA - no one seems to know whom to contact, or indicating wrong e-mail addresses. Concerning the data transfer of this table, I found: 1) The correction factor f in this table indicates, that the refraction values are for the condition of the standard atmosphere at the indicated height. Standard temperature at height is 15?C minus the lapse rate of -6.5?K/km. As a result of this it was also assumed that the standard pressure at height corresponds to the one of the standard atmosphere for the indicated height. 2) The position of the refraction values between the lines, indicate, that the refraction for the value shown has to be interpolated from the altitude values in the line above and below. 3) In order to compare the results with e.g. the values of Bennett's approximation formula, the table 6 refraction values had to be converted into refraction values for standard sea level condition. (I refer to these converted values as the normalised data.) Note: For comparing the data I used Bennett's original formula. Preliminary results: 1) The polynomial fit of these data would already be a basis for programming a computer function. 2) The refraction values in the table for a height of 0m (sea level) correspond well with those of Bennett's formula. 3) The normalised refraction values show, that the influence of the height (a.s.l.) vary for an altitude of 0? in the range of approx. 34' (for a height of 0m) to approx. 38' (for a height of 10'668m =35'000ft). 4) The influence of the height is diminishing with increasing altitude. In the altitude range of +3? to +6? the maximal deviations of the refraction values are about 1' (corresponding to about 10%) to those at 0m height. In the mean time I also found the source code of a BASIC program to calculate refraction by integration. The program was described in Sky & Telescope of March 1989. Without having the original article, I transcribed the program into the language with which I am working at the moment, i.e. in Pascal/Delphi. A comparison of the refraction values, either from the table 6 or those from Bennett, with those of the program show that those depend substantially of the selected refraction index of air. The problem of calculating the refraction becomes now a problem of calculating a realistic refraction index for air, which depends on the wavelength, temperature, humidity. This integration program seems to me useful for calculating or checking individual values. But when it comes to calculate large amounts of refraction values, a good approximation formula would be more advantageous. This is actually the reason why I might come back to a procedure for good estimations of refraction values. All this investigations done so far are for refraction values for APPARENT negative altitudes. For my program I need however also the "inverse", i.e. the calculation of the refraction for physical, TRUE negative altitudes,which has not been tuched so far. As you may see, this table 6 became very helpful. With the help of it I came some steps further, but there seems still to be quite a long way to go. Marcel