# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Real accuracy of the method of lunar distances**

**From:**Jan Kalivoda

**Date:**2003 Dec 30, 19:24 +0100

I have sent a message about this theme two weeks ago. Compelled by the question of Fred Hebart and after reading the thread about the accuracy and the precision started by Kieran Kelly, I return to it once more. Lunars are coming back again and again in our discussions and are treated from many points of view in the group. But the question of their attainable accuracy in the real conditions AT SEA emerged only sporadically and only on the basis of several isolated practical trials of group members, if I can remember. I have gained the copy of a German paper on this subject. It was published in "Annalen der Hydrographie und maritimen Meteorologie", 1889, p. 156 -163. (It was the periodical of the German Coast Guard for long decades.) The author was Fr. Bolte, the member of the Marine Observatory in Hamburg. I have some later papers on various subjects of the nautical astronomy from him and I appreciate him much. In the years 1887-88 he made two sea voyages to South America and Australia (as an apprentice nautical astronomer beginning his career, I guess) and collected his own observations made onboard to ascertain the attainable accuracy of various contemporary methods of the celestial navigation. He tried the star observations above the twilight and night horizon above all, but he also observed and recorded 34 lunar distances during these two voyages. According to positions given for his star observations, he travelled by a steamship and not by a sailing ship. It must be said that Bolte wasn't an opponent of lunars. To the contrary, he enforced them as the logical method of checking chronometers errors (not rates) during long trips of sailing vessels and he published the new approximate method for clearing the lunars in 1894 that could attain the accuracy of some 5" of the reduction. Of course, the doom of sailing vessels and the practice of carrying several chronometers by steamships exterminated the lunars before Bolte's eyes and maybe even earlier - long before radio signals came. Back to his observed lunars. Unfortunately, he doesn't state the method of their reduction. He compares his results with the chronometer values corrected by actual chronometer errors obtained backwards by interpolation from the errors ascertained before starting the deep sea leg of the voyage and afterwards in harbors. One cannot imagine a better method for that purpose before the arrival of radio signals. (If one excludes the possibility of using a dedicated ship cruising in sight of an ocean rock with the longitude accurately known - the observations near the coast could have been distorted by the irregular refraction, so frequent in coastal areas.) The observations were made by the sextant with the telescope magnification 9.5, tested by his Marine Observatory as "good", not "excellent". Each lunar observation was averaged from six shots. Bolte gives some data for each his observation (dates, individual errors, distance body as "Sun/star"). Time errors obtained by comparing the averaged result of every lunar observation with reduced chronometer times (see above) were converted to angular errors of observation in arc-seconds, based on the actual velocity of the Moon in her right ascension. Bolte also compared his results with another series of 82 lunars, made by some captain Behrens onboard a brig sailing from Java to England around the Cape of Good Hope in 1886. No details are given for this series, only summarized errors. I give these Behrens' errors in parenthesis below. Here follow Bolte's and Behrens' "probable errors" ("wahrscheinliche Fehler"): 31" for one distance (averaged set of shots) of the Moon and a star (18" by Behrens) 22" for one distance of the Moon and the Sun (18" by Behrens) 9" for the average result from two distances measured simultaneously eastward and westward from the Moon (13" by Behrens) (9" by Behrens for two distances taken shortly before and after the same new Moon; Bolte hadn't enough data to find this value) Notes: The first question is, what is meant by the term "probable error" ("wahrscheinlicher Fehler"). At the beginning of the paper, Bolte says (in translation): "I had established the differences between true positions of ship (for lunars, GMT values obtained by the chronometer and its backward interpolated error should be understood - J.K.) and observed values and I have computed the probable errors by the method of the least squares". I don't pretend to instruct anybody in the group in statistics and in the theory of errors. But I understand Bolte's term "probable error" (PE) as the error limit, which is exceeded in 50% of observations according to the theory of errors. It is found by multiplying the standard error (SE) of the observations set by the factor 0.6745. The SE itself can be found by the formula: SE = square root of (sum of squared errors of measurements against the correct values / number of measurements) So as to verify my assumption, I computed this probable error for all sets of observations in Bolte's paper (not only for lunars) and I always obtained the same values as given under the name "wahrscheinlicher Fehler" by Bolte. Accordingly, I consider Bolte's error values to be probable errors as stated in the previous paragraph. ---------------------- We can see that Bolte's PE's for Sun and for a star as a distance body differ substantially. But if we remove two outliers (errors of 120" and 127", both for a star as a distance body) from the set, we obtain 20" as the PE for stars, roughly the same value as the PE for the Sun and corresponding with Behrens' values. Bolte says that these two outliers were taken in two subsequent days with the very rough sea. Accordingly, we can maybe assume the value 20" as the PE of lunars generally for these two series of observations (34/82 items) of Bolte and Behrens. ----------------------- The PE given for the average of two lunar distances taken eastward and westward from the Moon is less than half the value for one distance of both. If only random errors would be present, this relation would be less, 1.41 as the square root from 2. Therefore, it is evident that systematic errors hide in both sets. (This fact undermines all our statistical considerations a bit, but what shall we do?) These systematic errors (small index, collimation, parallelism errors of the sextant, which cannot be removed by any adjusting) hopefully tend to cancel themselves partially, when taking the distances to both sides from the Moon. Such procedure of taking two lunars eastward and westward from the Moon simultaneously was recommended in handbooks exactly for this purpose from the very beginning of lunars. Therefore, one easterly and one westerly lunar distance at least were tabulated in almanacs from their first editions (four or six were given later, as both distances taken should be approximately the same for best results). But I wonder if one of lunarians of ten followed this advice. (And among navigators, one of five or so was lunarian, isn't it?) ------------------------ Now let us consider the maximum errors of lunars that should be taken into account, based on the values given above. If you multiply the standard error (SE) by 2, you obtain the error that should be not exceeded in 95% of observations, and you can be statistically sure that your error won't exceed the value of (SE times 3) in 99,7% of observations. Therefore, we can accept the maximum possible error in practice as SE ? 3 = (1/0.6745 ? PE) ? 3 = 4.5 ? PE very nearly. For lunars, PE of 20" times 4.5 gives 90" = approximately 180 seconds of time = approximately 45 minutes of longitude (the exact value depends on the actual velocity of the Moon in R.A.). For finding your position at sea, you should accept this maximum value in both E and W direction and your real position lies somewhere in the arc of longitude 1,5 degree long. This makes lunars nearly unusable in practice in my eyes - where I make a mistake? Of course, when you take two lunars to the East and to the West from the Moon simultaneously, the maximum possible error drops to the usable limits (onboard sailing ship). ------------------------- It should be noted that the errors of lunar tables were negligible compared with the errors of measurements only after cca 1880 (after Newcomb). From cca 1820 to 1880 (from Damoiseau and others to Newcomb) one had to accept the error of 20-30" in lunar tables and from 1755 to 1820 (from Mayer to Damoiseau) the possible error was 60" . Maybe this fact gives a commentary to the quotation from Cotter (A History of Nautical Astronomy, p.256), who himself quotes the report of Parry's Arctic expedition (1821-1823) concerning lunars taken from the ships caught by ice and staying at the same place for many months (not drifting!): "The mean of 2500 observations in December differed 14' from the mean of 2500 observations in the following March; ..." ================ I want to be refuted, as I like lunars very much. Jan Kalivoda