# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Slopes and least squares**

**From:**George Huxtable

**Date:**2010 Dec 9, 12:00 -0000

Thanks to Lars for expressing this matter so clearly. I have just been pondering the same thing myself, and had arrived at exactly the same conclusion, but Lars got there first, explaining it much better than I would have done. So, just like Antoine, I'm pleased to back up Lars' conclusions. George. contact George Huxtable, at george{at}hux.me.uk or at +44 1865 820222 (from UK, 01865 820222) or at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK. ----- Original Message ----- From: "Lars Bergman"To: Sent: Thursday, December 09, 2010 11:03 AM Subject: [NavList] Slopes and least squares There have recently been some discussions regarding evaluation of observational data with the help of a straight line of "best fit". I haven't followed the discussions in all details, but have some comments on one issue that possibly isn't well known. Given a set of data pairs (times and altitudes (or distances)), it is possible to calculate a line of best fit using the method of least squares. This line gets a certain slope and intercept, that minimize the sum of the squared distances between the observed values and that line. Now, if you calculate the average value of times and the average value of altitudes, this data pair is placed exactly on this line. This is a mathematical fact (pointed out by Alex E a couple of years ago, on this list). Thus there is no reason to calculate and plot the line of best fit in order to find a "better" value for sight reduction, just use the average of times and altitudes. Furthermore, if you want to use a pre-determined slope, that you know your observatinal series should follow, then this line with a given slope also passes exactly through the point of average time and average altitude, irrespective of slope, when adjusted to minimize the sum of squares. This line actually pivots around the "average point". As soon as you move the line off the "average point" the sum of squares will increase, minimum sum is obtained when using the "least square slope" mentioned in previous section. Thus there is no reason to calculate the expected slope either, just use the average of times and altitudes. Detecting blunders can be done by inspection. Lars, 59N 18E ---------------------------------------------------------------- NavList message boards and member settings: www.fer3.com/NavList Members may optionally receive posts by email. To cancel email delivery, send a message to NoMail[at]fer3.com ----------------------------------------------------------------