Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Re: Accuracy of position (sextant error simulation)
    From: Jim Manzari
    Date: 1999 Oct 25, 8:55 AM

    George --please read my remarks interspersed below...
    George Huxtable wrote:
    > This is a response too a posting made earlier today by Jim Manzari.
    > His analysis illustrates the dangers in using statistics to draw
    > conclusions from unrealistic and incorrect assumptions.
    Before this thread gets too far off course due to your pejorative remarks and
    in the interests of education, which is the goal of this mailing list, let's
    make clear that my original post in this thread was intended to answer
    Geoffrey Kolbe's specific question:
        ...you say, "Right, that is what it is." Or, as an old sea salt do you
        say, "That's all very well, but in my experience, the actual altitude may be
        up to x' either side of that, so I will just keep that in mind when plotting
        my position."  The question to the old sea salts is, how big is x?
    It is common practice, when faced with such a question, to model the errors in
    a systematic way to develop an overall error budget.  In my view, the
    principle non-systematic sextant-related errors are...
       - Failure to align the sextant with the local vertical
       - Random or non-systematic errors in the sextant itself
       - Failure to find the correct horizon or height of eye
       - Failure to apply the correct refraction correction
    There are other sextant-related errors depending on the celestial object
    observed, but I don't know how to model them.  The sun, moon, and planets
    suffer from several optical phenomenon that effect accuracy, such as
    irradiation, phase correction, glare, and limb darkening.  These type of errors
    have much to do with the eye-sight or physiological capability of the
    observer, therefore I did not attempt to model these kinds of errors.
    Before going further it should be emphasised that sextant-related,
    non-systematic errors are truly _random_ in nature and can only be described
    in a statistical way.  You may have a personal prejudice against the use of
    statistics to evaluate the scope of these errors, but your personal feelings
    won't change the fact that the errors are probabilistic.  Indeed, Dutton's
    Navigation and Piloting has this to say...
      "The novice observer will find that his sights do not yield satisfactory
      lines of position...These sights should then be plotted on a large sheet of
      plotting paper...A "line of best fit" is then drawn through the string (of
      observations)...The divergence of the individual sights from this line will
      tend to indicate the magnitude of the observers random errors...The random
      error of a single sight is the greatest hazard to the accuracy of celestial
    > Let's deal first which what, according to Jim, is the "largest single error,
    > which dominates all others", and can give rise to an altitude error, in one
    > case, of -34 arc minutes.
    Perhaps I should have said..."the largest potential for error".  Of the four
    types of errors enumerated in the analysis, this is the one with the greatest
    span -- from zero (0) to -34 arc-minutes.  The other three types of errors
    have, at most, a span of only plus or minus 5 arc-minutes maximum.
    It is important to point out that the -34 arc-minute error occurred only once
    out of the total 1000 simulated observations.  To model this error, I assumed
    a normal or Gaussian distribution for the mis-alignment angle with a mean or
    average value of zero and a variance of 5-degrees.
    > This depends entirely on what Jim has arbitrarily chosen as a reasonable
    > scatter for his angle phi, the tilt of the sextant from the vertical. In
    > that worst case of -34 minutes, my estimate of the tilt that produced it is
    > about 8 degrees from the vertical (it depends slightly on what the actual
    > altitude was for that measurement).  No wonder tilt dominates all other
    > errors!
    For some reason you have focused on the single most extreme occurrence of
    vertical alignment error and have ignored the average error and its standard
    deviation.  The average for this type of error was 1.8 arc-minutes with a
    standard deviation of plus or minus 3 arc-minutes.  The average value should
    not give much cause for concern, but I'm surprised that you did not find the
    spread or standard deviation of this error "dominating".
    The following table shows the distribution of the vertical mis-alignment error
    (binned in 5 arc-minute bins)...
          Ve       # of
      (arc-min) Occurrences
          0.00     780
          5.00     174
         10.00      29
         15.00      11
         20.00       4
         25.00       1
         35.00       1
    > What navigator would get a sighted body to kiss the horizon, and
    > then end up with an altitude corresponding to the value he would have got
    > with 8 degrees tilt, with enough error to put the upper limb of the Sun
    > where the lower limb ought to be? The notion is absurd. I ask Jim to state
    > how he has defined the scatter of sextant tilt, and on what basis he has
    > chosen that scatter. Is it realistic?
    You would know, if you had any real offshore experience, a navigator must
    frequently grab a quick sight of a star when it pops out from behind a cloud.
    In the tropics, for example, twilight lasts about 15-minutes and low, fast
    moving clouds are the norm.  The navigator must work quickly.  There is
    frequently no time to swing the sextant in the careful manner that is possible
    when working in your backyard or on a shoreline.  When using an artifical
    horizon for practice, this problem of vertical alignment never happens.
    The simulation was intended to model the "randomness" of a real navigator at
    sea.  Indeed, I think the simulation may have been overly optimistic.
    According to the simulation less than 50 of the observations would have an
    error of 10 arc-minutes or more.  How many of us "old-salts" can say that we
    are consistently this good?  The reason for multiple-star LOPs is to remove
    some of this "randomness".  I normally shoot 6 stars and retain of the best 3
    or 4 lines.  In rough seas and cloudy weather I often see lines with errors of
    10, 20, 30 or more miles due to the random nature of the sight-taking process.
    > Another factor Jim has considered is sextant index error. This plays no
    > part whatsoever in the altitude error as long as the index error has been
    > checked and allowed-for (the easiest thing in the world to do). I
    > interleave index checks between each sextant sight, for my plastic sextant,
    > though it's not necessary to go that far. The index-error contribution to
    > altitude error is ZERO.
    It is wrong to assume that all the index-type error has been removed.  My
    guess is that you can not see this error in your plastic sextant, because it
    lacks the necessary precision.  The Plath sextant that I use has done good
    service for more than 25 years and when last checked by the manufacturer is
    accurate to within 12 arc-seconds (0.2 arc-minute) _anywhere_ on the arc.
    Also I've noticed that if I want 0.1 arc-minute accuracy when checking for
    index error, it depends on which direction I approach the reading when turning
    the adjustment knob.  In addition, and there is always some random error in
    interpolating the graduations of the vernier.
    A simulation of 1000 sights shows that this error averages to be zero, but
    could be as large as plus or minus 0.7 arc-minutes in 33% of the observations
    (normal distribution with mean of 0 and variance of 0.5 arc-minutes).  It is a
    small error, but it does add into the total error budget of the sextant and,
    in my view, must be included in any meaningful study of observational errors.
    > Jim has considered refraction of the light from the body, and correctly
    > found that at angles above 30 degrees, variations are negligible.
    This conclusion, George, is completely incorrect.  The simulation attempts to
    model the error in the _refraction factor_ (f) caused by a difference between
    actual temperature and pressure at the time of the observation from the
    standard temperature of 10C and standard pressure of 1010 milli-bars.  It has
    nothing to do with refraction as a function of elevation angle.
    George, I suggest you open a Nautical Almanac to the first few pages.  You
    will see a special table designed to adjust refraction for non-standard air
    temperature and pressure.  Standard refraction corrections assume a
    temperature of 10C (50F) and a barometric pressure of 1010mb (29.83 inches).
    What is the error introduced by ignoring this correction?  That was the
    purpose of the simulation, not as you say to determine that refraction varies
    with elevation angle.
    > But he has ignored the unpredictable changes in the dip caused by refraction
    > in the light path from the horizon, an important factor if not THE most
    > important factor. See my mailings on this topic earlier today.
    Yes, I have ignored mirages, etc, just as I said I would in the introduction
    to my original post and in the paper itself.  And for good reasons...(1)
    Bubbles (mirages) in the air mass close to the horizon are relatively rare,
    except when the sea and air temperature are markedly different, or when sights
    are taken over land, or in a few locations in the world such as the Persian
    Gulf; (2)I don't know how to model this kind of rare event.
    George, if you had any real experience working as a navigator you would
    probably know that conditions giving rise to air mass non-uniformity near the
    horizon are relatively easy to detect.  Experienced navigators are sensitive
    to this problem whenever the air/sea temperature differential is large,
    whenever the sun or the moon appear distorted when rising or setting, or
    whenever a sight is taken across a land mass (island or peninsula).
    I suggest you spend some time studying a copy of the old fat Bowditch or
    Duttons.  Both these books have a long discussion of this subject.  In my
    view, this type of error is too rare (and too easily detected) to consider in
    an general study of sight errors.
    > I could go on, but I think I've said enough.
    Well, George, perhaps it is good you did not go on.  Otherwise, I would feel
    compelled to continue answering your mistaken conclusions lest they be
    accepted by default.
    Jim Manzari

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site