# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: short dip**

**From:**Thomas Alley

**Date:**1997 Jan 03, 20:14 EST

>> Nautical Almanac. The formula is Ds = 0.4156*d + 0.5658*h/d, > >Where do those magic numbers come from? I hate not knowing that sort >of thing... > > Geoff Kuenning g.kuenning@XXX.XXX > http://fmg-www.cs.ucla.edu/geoff/ The formula above is actually a simplified version of the following: ( h Bo d) Ds = 60 atan (-------- + ----) (6076.1 d 2 r ) where: h is the height of eye above the sea surface, d is the distance from the observer to the waterline being used as a "horizon" (in nautical miles), Bo is a parameter which describes "typical" atmospheric refraction, usually taken to be 0.8321 (don't ask me what the units are!) r is the mean radius of the Earth in nautical miles (3440.1). 6076.1 is the number of feet in a nautical mile. Starting with the second term, we can simplify it as follows: Bo d 0.8321 d d ---- = ---------- = ------ 2r 2 * 3440.1 8268.5 For the distances we are talking about (i.e., less than 5 nmi), this will always be a small number (< 0.0006). As our "horizon" gets closer, this term will get smaller yet. Since this term is dealing with atmospheric refraction, it makes sense that its effect should diminish as one looks through less and less air (i.e., as the object being observed -the horizon- gets closer). In the first term, the "6076.1" in the denominator is basically converting the height of the observers eye from feet to nautical miles, thereby keeping the units consistent within the equation. If we rewrite the first term using H (height of eye in nautical miles), it reduces to H/d, or the ratio between the observer's height of eye and the distance to the "horizon." Forgive the ASCII "sketch", but it's the only medium available... Let's draw a picture of what we have: | . (dots on diagonal) | . H | . | . a |_____________________ d I've labeled the angle between the water and the observer's line of sight "a". If you recall your high school trig: tan(a) = H/d The angle, is therefore the arctangent of the ratio: a = atan(H/d) But the answer is given in degrees. To give an answer in minutes, we would multiply by 60: a' = 60 atan(H/d) It's starting to look like what we started with, right? Just a little further now.... Since we are usually dealing with eye heights of less than 20 feet (0.003292 nmi) and distances of at least 0.5 nmi, it would be safe to say that our ratio of H/d will almost never exceed: 0.003292 / 0.5 = 0.0064 (approx.) For small ratios of H/d, the arctangent function can be approximated with the following formula when we are working in degrees: 180 x atan(x) = ----- = 57.3 x pi Let's take what we learned so far and substitute it into the equation we started with: ( h Bo d) ( h d ) Ds = 60 atan (-------- + ----) = 60 * 57.3 * (-------- + ------) (6076.1 d 2 r ) (6076.1 d 8268.5) 3438 h 3438 d = -------- + ------ = 0.5658 (h/d) + 0.4158 d 6076.1 d 8268.5 Rewrite and compare with what you wrote above: Ours: Ds = 0.4158 d + 0.5658(h/d) Above: DS = 0.4156 d + 0.5658(h/d) Pretty close (within 0.1%). The differences are basically the degree of precision that I chose to use in approximating the arctangent function. Now after all of that: Where do the numbers come from? They are an amalgam of conversion factors for degrees to minutes, feet to nautical miles, atmospheric refraction coefficients and the radius of the Earth. It's probably more of an explanation than you wanted, but I hope it shed some light on how all of this works. Tom ------------------------------------------------------------------------ Thomas Alley alley@XXX.XXX KD2VH, Youngstown, New York Alberg 35, "Tomfoolery"