# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Resume of "Averaging"**

**From:**Chuck Taylor

**Date:**2004 Nov 6, 15:47 -0800

Alexandre Eremenko and Herbert Prinz have been discussing the averaging of celestial sights. The following is an attempt to summarize: Consider the following hypothetical situation (the actual numbers here are made up): Body Time Altitude Spica 20-00-00 25d 10.5' Spica 20-01-00 25d 05.1' Arcturus 20-03-00 35d 30.0' Arcturus 20-04-00 35d 25.0' Arcturus 20-05-00 35d 20.0' Arcturus 20-06-00 35d 15.0' Arcturus 20-07-00 35d 10.0' Dubhe 20-09-00 42d 00.0' We have 2 observations of Spica, 5 of Arcturus, and 1 of Dubhe, for a total of 8 observations. How should we proceed in obtaining a fix? One obvious solution is to simply choose one observation of each body, then reduce and plot. The fix would be plotted somewhere within the "cocked hat". Alex would argue that we could do better than that by averaging the two sights of Spica (treating the average time and the average altitude as a one sight), averaging the 5 sights of Arcturus, and taking the single sight of Dubhe. (The motivation for averaging is the expection that random errors will cancel out in the process of averaging.) If I understand Herbert's objection correctly, he would claim, "Now wait a minute. You have 5 sights of Arcturus, but only 2 of Spica and 1 of Dubhe. How can you give the same amount of weight to the single sight of Dubhe as you do to the 5 sights of Arcturus? There is more information in the 5 sights of Arcturus than in the 1 sight of Dubhe, yet you are giving them the same weights." (I hope Herbert will forgive me for putting words in his mouth. If I have misunderstood him, I apologize in advance.) In theory you can only justify giving equal weights to each set of observations if each set contains an equal number of sights. In practice, you will probably not go far wrong anyway. Nor will you go far wrong in practice by selecting one sight from each set ("run") of sights. I happen to have a copy of the paper, "On the Overdetermined Celestial Fix", by Thomas and Frederic Metcalf. (Tom was once a contributor to this list. The last I heard, he was associated with the Institute of Astronomy, University of Hawaii, Honolulu.) I also have a copy of a follow-on paper, "An Extension to the Overdetermined Celestial Fix" by Tom Metcalf (alone). I obtained my copies of these papers some years ago directly from Tom. At the risk of gross oversimplification, I present the following analogy: Suppose you have two equations in two unknowns, each representing a line in a plane. Assuming the two lines are not parallel, the two lines intersect at a single point, and that point can be found by solving those two equations simultaneously. The solution can be by one of several methods, including Gaussian elimination, Gauss-Jordon elimination, or by the use of matrix algebra. Now suppose that we have 3 equations in two unknowns. We observed above that a unique solution is determined by two equations in two unknowns. Now we have an "overdetermined" set of equations with no unique solution. This is where the method of least squares enters in. We find the point such that the squared distance between that point and each of the lines in minimized. As Herbert pointed out, we can do this using matrix algebra. Now, let us return to the example of our sights of Spica, Arcturus, and Dubhe. Herbert's argument is that we ought to compute 8 lines of position (2 for Spica, 5 for Arcturus, and 1 for Dubhe) and find that point which minimizes the sum of the squared distances between it and each of the 8 lines of position. That would be solving an overdetermined celestial fix by the method of least squares. With this method, each observation from each body is given equal weight. Herbert, have I stated your argument fairly? The Metcalf papers actually address a slightly different form of overdetermined celestial fix. Quoting from the earlier of the two papers, "... using these methods, an accurate fix can quite often be determined from 10 to 20 observations of a single celestial body spanning only 10 to 20 min of time." The algorithm uses a Lagrange multiplier and the simultaneous solution of a nonlinear equation for the Lagrange multiplier and a number of linear equations. The solution is iterative. It's an interesting concept. Tom tried it out with "real observations of the Moon taken on land with an inexpensive plastic sextant, and with a pan of vegetable oil serving as an artifical horizon." He took 18 sights over approximately 35 minutes, and was able to confirm his known position within about 2.5 nautical miles. He programmed his algorithm on an HP-48SX calculator. The paper offers to make the code available to interested parties who send a stamped, self-addressed envelope. I'm not sure whether there was any expiration date on the offer, as it appeared in "Navigation: Journal of The Institute of Navigation", Vol 38, No. 1, Spring 1991. The follow-on paper appeared in Vol. 39, No. 4 (Winter 1992-1993). Best regards to all, Chuck Taylor Everett, WA, USA __________________________________ Do you Yahoo!? Check out the new Yahoo! Front Page. www.yahoo.com