Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Weighted least squares
    From: Peter Hakel
    Date: 2010 Dec 18, 13:45 -0800
    Peter Fogg wrote:
    "Rather than treating all points as equally valid, which averaging does, you provide yourself with a picture of your sights to assist you to choose which sights to favour over others, if any, and to what extent.  Perhaps based on your recollection of taking one of those sights while the platform was moving more than during the other observations, for example.  This is much of the beauty and appeal of this technique."

    and Gary LaPook wrote:
    "There has been some discussions lately about using the technique of
    graphing a number of sextant observations on the same body and comparing
    them with a line having a slope representing the change in altitude of
    the body over time to allow discarding erroneous observations so as to
    keep them from distorting an average of the good observations. If you
    are going to do this then, if you are not taking the sights from the
    beach but from a moving vessel, you must adjust the slope of the line to
    account for the movement of the vessel."


    The attached spreadsheet provides one way for averaging of sights.  Observation times go into column A and are considered exact.  Altitudes, which can be affected by inaccuracies, go into column B.  In cell F9 you specify the "Time of sight" for which the "averaged" altitude is then calculated (cells F10-F12).

    The procedure consists of three main steps:

    1) Altitudes from column B are adjusted for the motion of the observer and for the observed body's declination change. (The values in cells F1-F7 are preset to turn this step off, as this is not the main focus of the current discussion.)

    2) A basic (non-weighted) least-squares linear fit is calculated.  The quality of this fit is negatively affected by the single bad sight #5, which is quite literally out of the line established by the remaining "good" sights.  In cell I24 we see the result from this fit for the chosen instant of time being 29 as opposed to the "correct" value of 14.

    3) From the differences between the observed altitudes in column B and the corresponding values computed by the latest and so-far-the-best available linear fit we calculate the weighting factors for the next, now weighted, linear least-squares fit.  Thus values closer to the current fit have more influence in the construction of the next fit than the outliers do.  This is repeated until convergence; I hardcoded 6 iterations, which I hope will be sufficient in all cases.  The convergence can be judged by how close to zero the value in cell F14 is; this "Last diff" is the absolute value of the difference between the fitted altitudes (at the chosen "Time of sight") from the last two iterations.

    We can see that this process automatically eliminated the bad sight as the result in cell F10 shows the "correct" value of 14.

    The "Resolution" in cell F16 is not exactly an input parameter and its value is rather arbitrary, within reason.  Its main purpose is to avoid a potential division by zero in the weight factor calculation; it also provides a ceiling for the weight factors as a side effect.  I chose the value of 0.1' since that is the precision with which angular data are usually given in CelNav.

    Peter Hakel


    File: 114907.average.xls
    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site