# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Frank's formulas, was: Lunars: altitude accuracy**

**From:**Frank Reed CT

**Date:**2004 Nov 2, 20:10 EST

Alex E wrote:

"I verified Frank's formulas. They are indeed correct, and I have a rigorous proof of this. (Assuming no refraction)."

Great. I'm glad you looked into it. We might possibly both be wrong, but the odds are declining. ;-)

If anyone else would like to experiment with the math, the approach that I took was simply to differentiate the series expansion with respect to h1 and h2 (h1=Moon's alt, h2=other body's alt). The series expansion is

d' = d + dh1*A + dh2*B + higher order terms.

Details on this equation including definitions of A and B are on my web site under "Easy Lunars" (also in the list archives under the same subject back in April, 2004). If you take a partial derivative wtih respect to h1 and another w.r.t h2 and then assume an 0.1 arcminute max error in the cleared distance, you will find the equations that I posted previously (under the assumption that refraction can be ignored). There are bound to be different approaches to deriving these expressions, so it would probably be worth the time to crunch the math independently.

Note that it is not NECESSARY to ignore refraction. But doing so yields very short expressions which illuminate most of what is going on succinctly.

Incidentally, I also checked these results numerically, adding refraction in as well, and the conclusions I posted previously hold. I checked every 10 degrees of the Moon's altitude, and every quarter of a degree of measured distance and other body altitude. Makes a rather pretty graph if anyone is interested...

And Alex wrote:

"So indeed everything deteriorates when the distance becomes small"

Right. And this is a little wrinkle with short distance lunars that I think was well understood historically (from indirect evidence) but has not been known much in the modern "lore" of lunars.

And:

"but when the distance is near 90 deg, Moon's alt becomes irrelevant."

It's a miracle. ;-)

And wrote:

"Furthermore, when the altitude of the Moon or of the star is close to 90 d, this altitude

becomes irrelevant :-)"

One little thing here. Since the expression is really only true for the instantaneous derivative, the difference formulas for altitude error can't be taken too literally when the altitude is being considered. For example, if you plug in h2=90 degrees, the formula appears to say that any error in h2 (10 degrees? 30 degrees?) would be acceptable. But obviously that's not the case since at 80 degrees or 60 degrees altitude, the same euation permits a much smaller error in altitude.

This analysis raises another entertaining issue regarding the input data for clearing a lunar. We discussed the case of a lunar where the altitudes are each 45 degrees on opposite sides of the zenith and the measured distance is exactly 90 degrees. The difference in azimuth is 180 degrees, and the cleared distance is 89d 21.3'. If I shift the Moon's altitude to 40, the difference in azimuth is smaller: 147 degrees, but the cleared distance is still 89d 21.3'. The error in altitude has no effect. But what happens if I shift the Moon's altitude to 50 degrees? This is an interesting case because the observation is now *inconsistent*. There is no way to have a measured distance of 90 degrees when the Moon is at 50 degrees and the other body is at 45 degrees. But suppose that's what you've recorded. What happens? It is interesting that if you clear the distance, you will *still* get the number you're looking for: 89d 21.3'. But in this case, if you were to attempt to extract an actual value for the difference in azimuth, you would find a meaningless number (the intermediate step in the calculation gives a value for cosZ of -1.19). I find it rather entertaining that the clearing process is robust in this way and can handle inconsistent inputs.

And Alex concluded:

"Nice formulas, indeed!"

I was pleased the expressions were so "clean" when all was said and done. It didn't have to be that way...

Frank R

[ ] Mystic, Connecticut

[X] Chicago, Illinois

"I verified Frank's formulas. They are indeed correct, and I have a rigorous proof of this. (Assuming no refraction)."

Great. I'm glad you looked into it. We might possibly both be wrong, but the odds are declining. ;-)

If anyone else would like to experiment with the math, the approach that I took was simply to differentiate the series expansion with respect to h1 and h2 (h1=Moon's alt, h2=other body's alt). The series expansion is

d' = d + dh1*A + dh2*B + higher order terms.

Details on this equation including definitions of A and B are on my web site under "Easy Lunars" (also in the list archives under the same subject back in April, 2004). If you take a partial derivative wtih respect to h1 and another w.r.t h2 and then assume an 0.1 arcminute max error in the cleared distance, you will find the equations that I posted previously (under the assumption that refraction can be ignored). There are bound to be different approaches to deriving these expressions, so it would probably be worth the time to crunch the math independently.

Note that it is not NECESSARY to ignore refraction. But doing so yields very short expressions which illuminate most of what is going on succinctly.

Incidentally, I also checked these results numerically, adding refraction in as well, and the conclusions I posted previously hold. I checked every 10 degrees of the Moon's altitude, and every quarter of a degree of measured distance and other body altitude. Makes a rather pretty graph if anyone is interested...

And Alex wrote:

"So indeed everything deteriorates when the distance becomes small"

Right. And this is a little wrinkle with short distance lunars that I think was well understood historically (from indirect evidence) but has not been known much in the modern "lore" of lunars.

And:

"but when the distance is near 90 deg, Moon's alt becomes irrelevant."

It's a miracle. ;-)

And wrote:

"Furthermore, when the altitude of the Moon or of the star is close to 90 d, this altitude

becomes irrelevant :-)"

One little thing here. Since the expression is really only true for the instantaneous derivative, the difference formulas for altitude error can't be taken too literally when the altitude is being considered. For example, if you plug in h2=90 degrees, the formula appears to say that any error in h2 (10 degrees? 30 degrees?) would be acceptable. But obviously that's not the case since at 80 degrees or 60 degrees altitude, the same euation permits a much smaller error in altitude.

This analysis raises another entertaining issue regarding the input data for clearing a lunar. We discussed the case of a lunar where the altitudes are each 45 degrees on opposite sides of the zenith and the measured distance is exactly 90 degrees. The difference in azimuth is 180 degrees, and the cleared distance is 89d 21.3'. If I shift the Moon's altitude to 40, the difference in azimuth is smaller: 147 degrees, but the cleared distance is still 89d 21.3'. The error in altitude has no effect. But what happens if I shift the Moon's altitude to 50 degrees? This is an interesting case because the observation is now *inconsistent*. There is no way to have a measured distance of 90 degrees when the Moon is at 50 degrees and the other body is at 45 degrees. But suppose that's what you've recorded. What happens? It is interesting that if you clear the distance, you will *still* get the number you're looking for: 89d 21.3'. But in this case, if you were to attempt to extract an actual value for the difference in azimuth, you would find a meaningless number (the intermediate step in the calculation gives a value for cosZ of -1.19). I find it rather entertaining that the clearing process is robust in this way and can handle inconsistent inputs.

And Alex concluded:

"Nice formulas, indeed!"

I was pleased the expressions were so "clean" when all was said and done. It didn't have to be that way...

Frank R

[ ] Mystic, Connecticut

[X] Chicago, Illinois