# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Lunars: altitude accuracy**

**From:**Frank Reed CT

**Date:**2004 Oct 31, 22:27 EST

I wrote earlier:

"When shooting lunars, accuracy in the altitudes doesn't matter much. We've all heard that. But how much does it matter? I propose that you can get by with a 6 arcminute error for lunar distances around 90 degrees, assuming you want no more than an 0.1 minute resulting error in the cleared distance, but that generally the required accuracy (for constant altitudes) is proportional to the sine of the distance which means that for short distance lunars, the altitudes have to be significantly more accurate. This is yet another reason why short distance lunars would not have been popular historically."

I wrote this a week ago but it hasn't drawn any interest, so I think I should elaborate a little. In many postings on this list and in various publications and navigation manuals, people have variously written that the accuracy of the altitudes of the two objects used in a lunar observation don't matter very much. Usually, it's said that you can have 5 or 6 minute error in the measured altitude of the Moon or the other body and it will make no difference in the clearing of the observation. Try it. It's often true. But if you work the math, you'll find that this is true only "on average". Lunars can be much more sensitive to altitude accuracy in many cases. Lunars can be much less sensitive to altitude accuracy, dramatically much less sensitive, in a common situation which I'll describe in another post.

For short distance lunars, when the Moon is, for example, ten degrees from a star, the altitude of the Moon and star much be measured to be better than 1 minute of arc accuracy in order to insure that the resulting error in the clearing process is less than 0.1 minutes of arc. Fortunately, an equal error in both altitudes is much less important. An error in both altitudes might easily arise from a bad estimate of dip. If both altitudes are in error by the same amount, even up to 6 minutes or so, the resulting error will generally be less than 0.1 minute in the clearing process (unless the bodies are at altitudes of 10 degrees or lower). Still, getting the objects' altitudes consistently accurate to 1 minute or better requires great care in observations and this is another important reason why short distance lunars should be avoided. As a side note, in the latter half of the nineteenth century there was apparently some considerable interest in France in reviving lunars by focusing on short distances. But it was always emphasized that this procedure would require calculated altitudes instead of observed altitudes, and that means a lot more paper work.

I general, good (slightly approximate) expressions for the required accuracy of the altitudes are:

AccuracyBody = 6' * sin(Distance) / cos(BodyAltitude)

AccuracyMoon = 6' * tan(Distance) / cos(MoonAltitude).

I have never seen these expressions in print anywhere. Has anyone else encountered them?

The expressions above tell us how many minutes of arc error we can tolerate in the Moon's or other body's altitude in a lunar observation and not incur an error in the cleared distance of more than 0.1 minutes of arc. Clearly for short distances these accuracies are approximately proportional to the measured distance since sin(x) and tan(x) are both proportional to x when x is small. It's also worth noting that the required altitude accuracies are not as strict when the objects' altitudes are high (that's nice since higher altitudes are usually harder to observe with a sextant). Finally, I should note that these expressions are "slightly approximate" because they assume that refraction is insignificant. As as long both objects are above 10 degrees or so in altitude, that's a reasonable assumption. Also, these values are based on a mean value for the Moon's HP, and that's definitely a reasonable approximation

Frank R

[ ] Mystic, Connecticut

[X] Chicago, Illinois

"When shooting lunars, accuracy in the altitudes doesn't matter much. We've all heard that. But how much does it matter? I propose that you can get by with a 6 arcminute error for lunar distances around 90 degrees, assuming you want no more than an 0.1 minute resulting error in the cleared distance, but that generally the required accuracy (for constant altitudes) is proportional to the sine of the distance which means that for short distance lunars, the altitudes have to be significantly more accurate. This is yet another reason why short distance lunars would not have been popular historically."

I wrote this a week ago but it hasn't drawn any interest, so I think I should elaborate a little. In many postings on this list and in various publications and navigation manuals, people have variously written that the accuracy of the altitudes of the two objects used in a lunar observation don't matter very much. Usually, it's said that you can have 5 or 6 minute error in the measured altitude of the Moon or the other body and it will make no difference in the clearing of the observation. Try it. It's often true. But if you work the math, you'll find that this is true only "on average". Lunars can be much more sensitive to altitude accuracy in many cases. Lunars can be much less sensitive to altitude accuracy, dramatically much less sensitive, in a common situation which I'll describe in another post.

For short distance lunars, when the Moon is, for example, ten degrees from a star, the altitude of the Moon and star much be measured to be better than 1 minute of arc accuracy in order to insure that the resulting error in the clearing process is less than 0.1 minutes of arc. Fortunately, an equal error in both altitudes is much less important. An error in both altitudes might easily arise from a bad estimate of dip. If both altitudes are in error by the same amount, even up to 6 minutes or so, the resulting error will generally be less than 0.1 minute in the clearing process (unless the bodies are at altitudes of 10 degrees or lower). Still, getting the objects' altitudes consistently accurate to 1 minute or better requires great care in observations and this is another important reason why short distance lunars should be avoided. As a side note, in the latter half of the nineteenth century there was apparently some considerable interest in France in reviving lunars by focusing on short distances. But it was always emphasized that this procedure would require calculated altitudes instead of observed altitudes, and that means a lot more paper work.

I general, good (slightly approximate) expressions for the required accuracy of the altitudes are:

AccuracyBody = 6' * sin(Distance) / cos(BodyAltitude)

AccuracyMoon = 6' * tan(Distance) / cos(MoonAltitude).

I have never seen these expressions in print anywhere. Has anyone else encountered them?

The expressions above tell us how many minutes of arc error we can tolerate in the Moon's or other body's altitude in a lunar observation and not incur an error in the cleared distance of more than 0.1 minutes of arc. Clearly for short distances these accuracies are approximately proportional to the measured distance since sin(x) and tan(x) are both proportional to x when x is small. It's also worth noting that the required altitude accuracies are not as strict when the objects' altitudes are high (that's nice since higher altitudes are usually harder to observe with a sextant). Finally, I should note that these expressions are "slightly approximate" because they assume that refraction is insignificant. As as long both objects are above 10 degrees or so in altitude, that's a reasonable assumption. Also, these values are based on a mean value for the Moon's HP, and that's definitely a reasonable approximation

Frank R

[ ] Mystic, Connecticut

[X] Chicago, Illinois