Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Refraction at the horizon. was: Re: Celestial Navigation without a sextant.
    From: George Huxtable
    Date: 2008 Mar 13, 17:07 -0000

    First,  I should apologise for confusing, in Navlist 4635, between the works
    of two specialists on atmospheric optics, Brad Schaefer and Andy Young, in
    writing- "Brad Schaefer is a recognised authority on optics in the
    atmosphere, and more recently has produced an authoritative series of papers
    titled "Sunset Science"".
    I thank Marcel Tschudin for pointing that out. Thanks to Marcel for noting
    corrections to Schaefer's paper "Refraction near the horizon",  Publ.
    astron. society of the Pacific, vol 102, pages 796 - 805 (July 1990). I now
    have a .pdf copy of that paper, which I, too, could pass on to anyone
    showing an interest.
    Indeed, that allows me to put a bit of meat on to Meeus' bald statement-
    "the refraction at the horizon fluctuates by 0.3 degrees around a mean value
    normally". It turns out that the standard deviation was assessed to be 9.6
    arc-minutes, and Meeus' figure,equivalent to 18', was the range including 1
    standard deviation on either side of the mean, which should then embrace 68%
    of all observations. To include nearly all observations (95%) the overall
    range should be more than 38'.
    I recommend that anyone intending to enter into further discussion on this
    interesting matter should first acquaint himself with Schaefer's data, and
    what he has to say about it.
    Second, I suggest that Greg Rudzinski missed the important difference
    between refraction at and near the horizon, and that at much higher angles,
    when he wrote, in Navlist 4649- .
    " I base my 6 minutes of arc figure on the refration variables as
    seen in the A4 table of the Nautical Almanac. The temperature and
    pressure extremes are -7.3' to +6.9' from the mean."
    For highish angles (say 10 degrees and more) light is coming in at such an
    angle , through our atmosphere, which is a few km high, in such a way that
    the layers of air of different density are effectively plane-parallel. The
    curvature of the Earth can be neglected. In this flat-Earth approximation,
    any layer-structure of density changes in the air above us have no effect,
    in just the same way as light, travelling in and out of a slab of glass,
    resumes exactly its original direction. All that matters is the total change
    in the refractive index that the light-beam has met in its path from space
    (where the refractive index is exactly 1) to the observer.
    So for those high angles where that approximation is valid, the ONLY
    atmospheric effect of the refraction at that angle depends on  the
    refractive index of the atmosphere at the level of the observer's eye. And
    effectively, that means the air density at that level. The corrections,
    quoted in the almanac, are no more than an allowance for different air
    density, if temperature and pressure diverge from a nominal standard value.
    But at and near the horizon, things are very different, and those
    assumptions break down. Because of the Earth's curvature, the air-layers
    along the incoming light-path are no longer plane, but wedge-shaped, and the
    angle the density contours make with the incoming ray is changing
    contiuously. In that situation, each density-layer contributes its bit to
    the overall refraction, in a way that no longer cancels out in the overall
    The average affect of a standard atmosphere can be integrated along the
    light path, assuming a standard density variation with height (which is
    equivalent to assuming a certain temperature variation). That corresponds to
    the book value of horizontal refraction of 34 arc-minutes. But at these
    angles, it's highly sensitive to divergences from that standard temperature
    pattern, and at what height those occur.
    Yes, Greg can correct that 34 arc-minute figure for local air density if he
    wishes, but that is no more than a tiny fraction of the overall fluctuation
    in refraction at the horizon.
    Greg continues-
    "In reality on the
    ocean and from the beach I get better than + or - 4' from a GPS
    position. Extreme refraction can occure after the passage of a cold
    front. In this case opposing LOP's or a back sight will be needed to
    cancel out the refraction variable."
    I presume (and no doubt Greg will correct me if I am wrong) that he is
    referring here to bodies observed in the normal range of celestial
    observations (above several degrees in elevation) and NOT to observations
    made AT the horizon, which were what we were discussing.
    Next, to deal with some comments from Frank Reed, who wrote-
    | George, you wrote:
    | "What does Greg base his "6 minutes of arc" figure on, when it must
    | the uncertainties in refraction for the Sun, as seen on the horizon? How
    | well does he know what that refraction will be, and how much it might vary
    | from the "book" value of 34 arc-minutes?"
    | Strikes me as a 'not unreasonable' value. There is an awful lot of
    | variability within the variability. What I mean by that is that under
    | certain very common circumstances, e.g. at sea in temperate climates, the
    | day-to-day variability in the refraction at the horizon is relatively
    | (a few minutes of arc), but under other circumstances, which might be
    | for some observers, e.g. near the coast in cold weather, the day-to-day
    | variability in refraction could be much larger (tens of minutes of arc).
    And I reply-
    Greg's postulated value of 6 minutes of arc was indeed unreasonable, and
    based on a misunderstanding, as I have explained above.
    I ask on what evidence Frank bases his assertion that-
    | "under
    | certain very common circumstances, e.g. at sea in temperate climates, the
    | day-to-day variability in the refraction at the horizon is relatively
    | (a few minutes of arc), "
    And even if that assertion is valid, how on Earth does a navigator know,
    when trying to use a sunset time to ascertain his position, whether Frank's
    restrictions apply or not, within the "variation within the variability"
    that he conjures up?
     If we are talking about use of timing a sunset for real navigation,
    requirements for position knowledge can be very relaxed in mid ocean. It is
    only when a land mass is being approached that navigation becomes critical.
    So observations made at coastal sites, with a view over the ocean, will be
    particularly relevant in assessing the navigator's problem. And those are
    exactly the sites from which Schaefer and Liller have timed sunsets, at sea,
    in temperate climes.
    | You quoted Meeus,
    | " ... According to Schaefer and Liller, the refraction at the horizon
    | fluctuates by 0.3 degrees around a mean value normally, and in some cases
    | apparently much more"
    | And quoted Bowditch,
    | "Generally, the error in tabulated refraction should not exceed two or
    | minutes, even at the horizon" ...which you called 'absurd'.
    | The Bowditch statement is weak because it doesn't provide any ranges.
    Not just weak. Absurd.
    | The
    | refraction tables, when properly corrected for temperature and pressure,
    | in fact be counted on as exact for all practical navigation down to THREE
    | degrees altitude.
    Without disputing that assertion, I would like to see some evidence to
    support it. What precision is implied by "exact for all practical
    | Although some navigation manuals still recommend avoiding
    | sights below ten degrees altitude, this is over-cautious.
    | Meanwhile, claiming "two or three minutes" variability at the horizon
    | to be in conflict with the other source which suggests 18 minutes
    | variability in normal cases
     (incidentally, Greg's "6 minutes" is close to
    | the geometric mean of these other claims, for whatever that's worth ).
    Not much.
    | So
    | who's right? I think they're both right under certain conditions. As I
    | above, I think this is due to the great difference between the temperature
    | profile of the atmosphere on a large landmass (or near the coast),
    | especially in winter, and the temperature profile at sea. Extreme
    | temperature inversions are common near a continental coastline but rare at
    | sea. You can look at real data on this from daily weather balloons. Pick
    | island location like Bermuda, and you'll see much less variability than at
    | coastal or inland location.
    It's coastal conditions that interest the navigator, as explained above. And
    the Schaefer data calls for no "extreme temperature inversions"; we are
    talking about day-to-day scatter.
    | By the way, there are sometimes temperature inversions even at sea. Calm
    | weather especially is associated with them. It's worth knowing that the
    | conditions which would lead to anomalous low altitude refraction at sea
    | also often lead to "sea fog".
    Well, the sea fog is affected by temperature gradients in the lowest few
    metres, which would play a part in horizontal refraction, as well as dip,
    but the refraction of the incoming light depends also on conditions at
    higher levels, which sea fog tells us nothing about. If there's sea fog,
    then you can't see a sunset anyway. Can Frank reliably deduce, from absence
    of sea fog, that there's therefore some limit on the variability of
    refraction? If not, that comment is of little use. All Schaefer's
    observations were, presumably, made when there was no sea fog.
    | Finally then, what range in variability of the refraction should a
    | (or a celestial navigation enthusiast) use? I would recommend these:
    | 1) Trust the tables for altitudes of 3 degrees or above, but be sure to
    | correct for temperature and pressure. There's no reason to worry about
    | anomalous refraction above these altitudes.
    That might be usually true, but I have often seen a highly distorted Sun
    disc, at altitudes of well over 3 degrees, that leads me to question it as a
    general rule.
    | You DO have to be concerned
    | about anomalous dip, however, as a separate issue, no matter what the
    | object's altitude.
    | 2) Ignore tenths of a minute of arc in the refraction below 1 degree
    | altitude (so you can ignore the "improved" refraction tables published in
    | the Nautical Almanac starting in 2004 --they result from a relatively
    | trivial change in the atmospheric temperature profile).
    | 3) Where temperature inversions and other low-level variability are likely
    | to be small (a good distance from large landmasses, not becalmed, no sea
    | fog), expect less than six minutes variability in refraction even at the
    | horizon, probably less than three minutes.
    Evidence requested to support that assertion, please. How reliable is that
    proposed test to ensure that "inversions and other low level variability"
    are small? I understand (personal communication from Schaefer) that "our
    atmosphere always have rapidly and wildly varying thermal structure (i.e.,
    temperature inversions come and go on all time scales and are generally
    present at some level)." In that, he does not distinguish between over-land
    and over-sea, though no doubt such effects are significantly greater over
    | Also, under these same
    | conditions, anomalous dip is likely to be small.
    | 4) If there is any reason to believe there is an unusual temperature
    | in the lower atmosphere (particularly close to shore in cold weather,
    | especially in the early morning), be aware that the refraction can easily
    | vary by 20 or even 30 minutes of arc.
    Half of Schaefer's events we are considering were sunrises, so presumably
    were in the early morning, and all were close to shore, in that all
    observations were made from on shore locations looking to sea. So what
    conclusions can we then draw from Frank's advice?
    The part of the path of incoming light that gives rise to such variations
    extends to 100km or so from the observer
    | Incidentally, I base these conclusions, in part, on a large number of
    | refraction integrations that I ran back in 2005 (I think) when Marcel got
    | interested in the production of refraction tables. It's quite
    | straight-forward to take an observed temperature profile and generate a
    | refraction table just for those special conditions.
    No doubt it is, Indeed, Schaefer himself published such an integration
    procedure (which I haven't seen) in Sky and Telescope, 77, 311 (1989), and
    the procedure Frank used may have been that one, or one based on it. It was
    to test the unexpectedly large fluctuations that it predicted, that
    Schaefer's survey of horizontal refraction was undertaken.
    Schaefer's survey involved 116 observations at 4 different sites, worldwide.
    If Frank PREFERS simulation evidence from integrating observed temperature
    profiles, to such direct measurements, then perhaps he will explain to us
    the statistics of the population on which his conclusions were based. Just
    how many temperature profiles did he consider? What was their geographical
    distribution? How many were taken at the relevant times of sunrise/set? How
    many measured air temperatures some tens of miles out from the coast,
    relevant to Schaefer's survey and to navigators' needs? And what was the
    resulting scatter that Frank calculated?
    | Of course, navigators at
    | sea do not generally have access to current atmospheric temperature
    | profiles. Some naval navigators might have been able to observe the real
    | temperature profile (by launching balloons or even aircraft), but the
    | computing power to generate refraction tables "on the fly" probably
    | too late for naval celestial navigation. And again, it's totally
    | for altitudes above three degrees, so the value of such tables is limited
    | any case.
    What would be particularly interesting would be a measurement of horizontal
    refraction and temperature profile at the same place and time, but I doubt
    if that's ever happened.
    In navlist 4617, Peter Fogg suggested-
    "Here's a thought: if the time of apparent sunrise/sunset was observed
    the extent of the difference or inaccuracy shown by observation
    compared to calculated data could be evaluated on a regular basis and
    contrasted with other information about position."
    And indeed, that procedure might well add useful information to Schaefer's
    body of data on horizontal refraction, for locations taken far from land. On
    those occasions when the sky was clear right down to the horizon, with no
    sign of distant cloudbanks, the time of the last (or first) glimpse of the
    Sun could be logged to the GMT second, from a well-known height of eye, with
    a precise GPS position at that moment, with a note of the datum in use. With
    the closeness of horizon that can be seen from a small craft, it could only
    work reliably in calm conditions without significant swell. How about that
    for a project on the next cruise, one that calls for no special
    instrumentation? It wouldn't be hard to collect that data together from a
    number of navlist members, and boil it down. Perhaps that's been done
    already, but if it has I haven't heard of it.
    contact George Huxtable at george@huxtable.u-net.com
    or at +44 1865 820222 (from UK, 01865 820222)
    or at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK.
    Navigation List archive: www.fer3.com/arc
    To post, email NavList@fer3.com
    To unsubscribe, email NavList-unsubscribe@fer3.com

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site