NavList:

Recent posts have questioned vehemently (2 Jan 13 per Frank Reed) or mildly (31 Dec 12 Alex Eremenko) whether it is or was possible to achieve particular degrees of accuracy and consistency with lunar distance observations. My own interest is in the instruments of navigation rather than the many uses to which they can be put. In 2011 I took up the challenge of the late George Huxtable to see how instruments perform in the real world at sea. I regarded recent statements as a challenge to see what can be achieved for lunar distance measurements in the best of conditions using the best of sextants. I say nothing about the quality of the observer except that I have spent a lot of amateur time peering through optical measuring instruments of various types, my vision can be corrected to 6/5 (it isn’t at present, as optometrists in New Zealand charge like wounded bulls for new lenses) and, as a copper-bottomed brass-bound , dyed-in-the-wool sceptic I do my very best not to “tweak” observations despite feeling tempted at times.

The sextant used was a 1978 Russian SNO-T sextant whose non-correctable errors in the range of angles measured were under 3 seconds, so I ignored them. As Frank Reed had sung the praises of a x7 telescope, I assumed he had used a prismatic monocular, so I did too for my first series, a vintage Beck Kassel 6 x 30 borrowed from a 1953 C Plath sextant. For my second and third series I used the Keplerian (“inverting”) telescope supplied with the instrument. The mirrors and shades are in perfect condition and, as I overhauled the instrument myself, I am confident that it is in as good a mechanical condition as possible. While backlash is well within the 6 seconds allowable by the specification, I consistently turn the micrometer drum in the direction of increasing the reading, both when estimating index error and when making observations. To minimise digit preference bias, I glued a paper vernier reading to 0.1 arcmin to the index.

Even quite large telescope collimation errors are of little importance in modern navigation, but when taking lunar distances one is trying to squeeze the last bit of accuracy out of the results and the angles measured are often large, when collimation error is most important. For example, if the telescope is mis-aligned by half a degree, the error at 105 degrees is about 20 seconds. The easiest way to check collimation is to use an engineer’s square to see that the face of the objective lens mount is square to the face of the sextant frame. You should not assume that it is safe to transfer a telescope from one sextant to another without checking this. There are several other, more-or-less sensitive ways of doing it. I had to place shims between the Beck Kassel mounting bracket and the body of the monocular to correct a large error, and by the time my patience had run out there was still a small collimation error remaining.

If the estimate of index error is wrong, then it is possible to have series of observations tightly clustered about a mean that is wrong. The results will have a small standard deviation (a measure of central tendency, or distribution of the results about the mean), implying good consistency of measurements, but the figure of interest, the mean, may be wildly wrong. For this reason, it is worth while taking great care with the index error. Though several people, Greg Rudzinsky most recently (6 Jan 13), have stated a preference for a particular method of estimation, when a few years ago I compared 30 observations each using the horizon, the sun and two stars, I could detect no statistically significant difference of the methods in my hands. On this occasion, I used the ridge of a barn on the horizon 6 km away. It is easier on the neck. For these and subsequent observations I clamped the instrument on top of a theodolite tripod. It makes a big difference to the ease of observations, especially in a 20 km/h wind gusting to 40 km/h. Incidentally, collimation error does not apply to index error estimations, as the reading is made at zero.

In the spread sheet file “Lunar 1”, Cell B25 gives the mean of 20 index error estimations and, for what it is worth, cell B26 gives their standard deviation.
Before making the lunar distance observation, I first achieved coincidence by holding the sextant in my hands. I used no shades and proceeded from the moon to the brighter object, the sun, using the sun’s glare as a guide to when I was approaching coincidence, at which point I put a dark shade in place. I then mounted the sextant on the tripod again and adjusted the index shades so that when the images overlapped, the moon was visible through the sun. I found it easier to determine contact of the limbs by bringing them apart. To minimise the effects of collimation and other optical errors, it is important to keep the objects as close to the centre of the field of view as possible.
I used Frank Reed’s lunar calculator to determine what the result should have been for the position and conditions, and the differences between the calculated and observed results are given in column H

Cells J13 and J14 give the means and standard deviation (SD) for ten observations with the monocular, while cells J23 and J24 give the results using the inverting telescope. The latter seems to give the better results and we do not need to do a Student’s t-test to see that there is a significant difference in the means of the errors. For a first attempt, these are not poor results.
As the following day was also fine, I did another set of observations, this time checking the index error using the same telescope, the inverting, that was to be used for the observations. These are shown in the file “Lunar 2”. Cell C25 gives the mean of 20 IE observations. The dispersion of results about the mean was somewhat closer than before. Cell G30 shows the mean error of 25 lunar distance observations, 0.032 arcmin or about 2 seconds, while the standard deviation in cell G31 tells us that about two thirds of the errors fall within the range of +11 seconds to -7 seconds.

Thus, by taking care with adjustment of a high quality sextant and averaging many results, it does indeed seem possible to achieve high accuracy on land with the instrument steadied on a tripod. The errors are likely to be much greater on land with a hand-held instrument and still more so at sea. The file “Cape St Diego” gives results of observations by the astronomer Richard Green aboard the Endeavour in January 1769. We can see that three sets of three observations were made and that readings were to the closest 30 seconds. Jesse Ramsden’s first dividing engine had been completed only two years previously, not to his satisfaction, and his second engine, which revolutionised precision angular measurement, did not become operational until about seven years later. It is likely that Green used a Hadley’s quadrant, probably of 15 inches radius, divided by hand with a vernier reading to half a minute.

To put these observations in context, the astronomer William Wales later wrote:”…the longitude…is near a degree too great; which is not at all surprising, if we consider, that although the air was extremely clear when these observations were made, yet the sea ran so high that it filled the quarter deck three times while they were observing and the motion of the ship was so great that Captain Cook did not attempt to observe.” My own errors using a metal sextant of 8 inches radius with a vernier reading to 30 seconds, made some time between 1790 and 1810, are shown at the bottom of file “Lunar 2”. The sextant was steadied on a tripod, no one was playing a fire hose on my legs and the ground was not rocking and shaking, yet my errors are of the same order as those of Green.

Bill Morris
Pukenui
New Zealand
----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------