# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Refraction near the Horizon ? Observation vs. Calculation**

**From:**John Huth

**Date:**2013 Apr 8, 09:42 -0400

Marcel -

Your numbers are probably closer to reality than mine. I'm just basing my guesstimates off of my limited experience and averaging. Yours probably take into account more conditions and observations.

Yes, urban environments are difficult, as I've found. I suppose there's not a lot of point in doing shots in an urban environment and expecting a lot of precision. There are all these little local climate issues.

On Sun, Apr 7, 2013 at 8:00 PM, Marcel Tschudin <marcel.e.tschudin---.com> wrote:

Hello John, You are welcome to jump in and give us a chance to share your view from a different perspective. I'm not surprised that you obtained considerable differences when determining the earth's radius from dip measurements. Because the radius is proportional to the dip squared, the error contributions from the dips will be doubled. Yes, measuring the dip by overlooking a metropolitan aera is not so ideal; a nearby hill overlooking the sea would be a better location. If I understand you correct, then the 1/5th is your rough estimate for k. This value could possibly be high or your thumb slightly thick. Surveyors used apparently 1/8th and the formulae for navigation use about 1/6th. A German Web-page on terrestrial refraction provides the following values: k = 0.125 during the day in undisturbed atmosphere, i.e. clear sky and normal viewing conditions. k = 0.20 all of the year during day and night with covered sky k = 0.30 during clear nights I am unfortunately not in a position to comment these values because I do not have any practical experience with them. Marcel On Mon, Apr 8, 2013 at 1:18 AM, John Hwrote: > ________________________________ > Hi, Brad, Marcel - > > I'll defer to Marcel on these things since he's generally thought longer and > harder than I have, but I'll just add some quick comments. I'd expect you > to get the reduced dip angle curve from using the radius of the earth > formula by about 1/5th or so. That's just a rule of thumb I use for > refraction. > > I actually tried to turn the process around with the help of some students. > We tried to measure the radius of the earth from dip angle measurements - we > used the top of the Prudential Tower in Boston. To establish a horizontal, > we took two containers of water separated by about 10 feet, and connected > with a hose. They then sighted down to the horizon. They also took other > dip measurements to nearby landmarks. > > When they went to calculate the radius of the earth, they were off by quite > some amount - I think it was a factor of two too large a radius. We went > through their error analysis, and I concluded that there was definitely some > large refractive effects going on that would throw you some distance off of > that Nautical Almanac data you attached. We tried to get a 'model > atmosphere' from the more nearby sightings, but I think that the kinds of > local heating one experiences in the city was problematic. > > I've seen this in surveying, when local sources of heat cause the images of > targets to jump around. > > I'm pretty sure you'll need a different correction for sighting celestial > objects, as light from them will travel a longer distance through the > atmosphere. You can take a limiting case - say your height is close to > zero - you'll get almost no dip correction at all, but the atmospheric > correction will be something like 34 arc-minutes on average for a star near > the horizon. The 34 arc-minutes is just some averaged value that will > clearly change, depending on conditions. But, maybe I misunderstood your > question on this point? > > In any case, this is all off the top of my head, and I'm jumping into a long > thread, so apologies if I'm wide of the mark. > > Best, > > John H. > > > > On Sun, Apr 7, 2013 at 4:22 PM, Brad Morriswrote: >> >> ________________________________ >> >> Hi Marcel >> >> Since the Nautical Almanac is not readily available to you, I took the >> time to enter Table A2 into a spreadsheet such that I could compare the >> noted nominal equations to the gold standard. The Nautical Almanac has dip >> as a function of HoE, and provides that to HoE of 48 meters. Presumably it >> ends there because this is beyond the nominal HoE for marine navigation. >> Consider, Jeremy (one of those who posts here, a navlist correspondant) >> captains a large merchant ship and his HoE has been given as 103 feet >> (~31.3meters). >> >> All equations, except one in particular are a very good fit to the data >> provided by the Nautical Almanac. >> >> The only equation that does NOT fit is dip=3438*sqrt(2h/R). You can see >> that this one equation yields a dip that is simply too large in the attached >> chart. To see what would make 3438sqrt(2h/R) fit the gold standard, I >> changed the effective radius of the earth until it did. Rather surprisingly >> when k=.164, the equation meets the gold standard! You can see that in the >> line 3438sqrt(2h/Rrefr). [Not shown is a graph of pure geometric dip, which >> happened to precisely meet 3438sqrt(2h/R).] >> >> To be sure I didn't misunderstand this equation, I referred back to >> Frank's post 122845 http://fer3.com/arc/m2.aspx?i=122845 and re-examined his >> dip0 equation, where he states "R is the radius of the Earth and of course >> dip0=sqrt(2h/R)" and then "multiplying by 3438 to convert to minutes of arc >> ". >> >> Instead of some snarky comments, I'll simply urge Frank to defend his >> post, and explain why ALL THE OTHER EQUATIONS MATCH THE NAUTICAL ALMANC FOR >> STANDARD ENVIRONMENTAL CONDITIONS EXCEPT FRANK'S APPROXIMATION. Dip for >> standard environmental conditions is simply NOT =3438sqrt(2h/R) >> >> ++++++ >> Now as to table A4. The instructions for table A4 are as follows: >> "The graph is entered with arguments temperature and pressure to find a >> zone letter; using arguments of this zone letter and apparent altitude >> (sextant altitude corrected for index error and dip [Table A2]), a >> correction is taken from this table. This correction is to be applied to the >> sextant altitude in addition to the corrections for standard conditions (for >> the Sun, stars and planets from page A2-A3 and for the moon from pages xxxiv >> and xxxv). >> >> Marcel, the top half is a graph of pressure versus temperature, with >> diagonal bands denoted by zone letters (see instructions). You find the >> intersection of your temperature and your pressure, yielding one of these >> zone letters. Next, you enter the table below, with the zone letter and >> apparent altitude (see instructions). There is clearly a row for altitude 0 >> degrees 0 minutes. The next row is for 0 degrees 30 minutes. Since our >> altitude is actually on the order of -3 to -12 minutes, it most closely fits >> with the first row. That first row shows corrections well beyond the temp >> and pressure corrections you derive from your method, as shown earlier in >> this discussion. That was the point I failed at making in an earlier post. >> The NA provides this correction table for non-standard conditions and the >> corrections are much larger than those provided by your method. >> >> I understand your argument that this table refers only to astronomical >> refraction (a body like the sun or a star), and not to terrestrial >> refraction (the horizon under dip). You may indeed be correct. I'm simply >> not sure. I assumed that the bending of light from the horizon is identical >> to the bending of light from an object whose light is just at that horizon. >> I may very well be wrong. >> >> ATTENTION TO NA EXPERTS >> Please define if the Table A4 is exclusively for celestial bodies, or if >> it can indeed be used for objects like the horizon under non-standard >> environmental conditions. Has anyone an equation for Table A4? >> >> Brad >> >> >> ---------------------------------------------------------------- >> NavList message boards and member settings: www.fer3.com/NavList> > > View and reply to this message: http://fer3.com/arc/m2.aspx?i=123420>> Members may optionally receive posts by email. >> To cancel email delivery, send a message to NoMail[at]fer3.com >> ---------------------------------------------------------------- >> >> Attached File: >> DipEquationsVsNauticalAlmana.pdf (no preview available) >> >> View and reply to this message: http://fer3.com/arc/m2.aspx?i=123418View and reply to this message: http://fer3.com/arc/m2.aspx?i=123424