Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Re: Astronomical Refraction: Computational Method for All Zenith Angles
    From: Frank Reed CT
    Date: 2005 Aug 22, 18:26 EDT

    [note to list members: I promise, no code in this message. there are some
    conclusions relevant to navigation in the last paragraphs]
    Marcel, you wrote:
    "It seems that there is something wrong in my stratospheric part. May I  ask
    you, Frank, to have a look at it? May be you can see what is going  wrong
    I am not using the analytic model from the Auer-Standish  article so I can't
    help you with your code. In my opinion, the authors' choice  of an analytic
    atmosphere model from 1944 was misguided. First, as I noted  previously, that
    model is simplified in a fashion that was determined by the  calculational
    limitations of sixty years ago. It uses the number "5" for the  polytropic index of
    the troposphere simply because it's an integer. Second, the  very idea that
    we need to treat "polytropic" and "exponential" sections of the  atmosphere
    separately is a limitation of the analytic treatment that obscures  the
    fundamental equivalence of these different sections of the atmosphere. The  whole
    atmosphere (assumed to be hydrostatic and spherically symmetric) can be  specified
    by giving temperature as a function of altitude based on real  empirical
    observations. If I say, 'temperature falls 7.2 degrees per kilometer  up to 11km
    and then remains constant above that height,' I have given enough  information
    to determine the atmospheric density profile and therefore the  refraction
    completely (apart from very small differences arising from changing  humidity at
    low altitudes and changing composition at very high altitudes).  Equally
    easily, I could feed the model a detailed temperature profile taken from  actual
    weather balloon observations with a hundred different lapse rates and  altitude
    limits. That's the beauty of this approach: it's completely  general.
    I was able to reproduce the sample refraction tables in the  Auer-Standish
    article almost perfectly. After a little more experimentation, I  found that the
    best results come from setting the lapse rate to -5.692 (degrees  Celsius per
    kilometer) below 11km and zero above that. Here's the zero observer  height
    0     2189.4   2189.4
    1    1532.6    1532.6
    2    1145.5    1145.5
    3     899.2     899.2
    4     732.7     732.8    0.1
    5     614.5     614.6    0.1
    10    330.5     330.5
    15    221.5     221.5
    30    104.0     104.0
    45     60.1      60.2    0.1
    In this table, the leftmost column is the altitude  in degrees, the second
    column is the refraction in arcseconds as calculated in  my code, the third
    column is the refraction listed in the Auer-Standish article  and the third column
    shows the difference between the two in arcseconds (when  different from
    zero). These differences are extremely small, and I consider them  insignificant.
    When the observer altitude is 15,000m, the agreement is  still excellent
    though the differences close to the geometric horizon are a  little larger:
    Tfactor of 1.0 and Lrate=-5.692 for obs h=15000  m:
    -3   2316.1    2316.4    0.3
    -2   1188.4    1187.9   -0.5
    -1    600.0     600.6    0.6
    0     353.3     353.4    0.1
    1     235.6     235.8    0.2
    2     171.3     171.5    0.2
    3     132.4     132.5    0.1
    4     106.9     107.0    0.1
    5       89.1     89.2     0.1
    10     47.4      47.5    0.1
    15      31.7     31.7
    30      14.9     14.9
    45       8.6      8.6
    I suspect that the small variances (all  still less than 1 arcsecond!) at the
    lowest altitudes are due to the small  differences in the way the troposphere
    is modeled. I don't think they're  anything to worry about.
    Next, the tables in the Nautical Almanac. Here  we have two versions:
    pre-2004 and post 2004. They require different lapse rate  models:
    Using a temperature factor of 273.15/283.15 and
    changing the  lapse rate to -7.25 reproduces a refraction table
    very close to the standard  (pre-2004) Nautical  Almanac:
    0.00       34.4   34.5    0.1
    0.25   31.4    31.4
    0.50   28.7   28.7
    0.75    26.4   26.4
    1.00    24.4    24.3   -0.1
    1.25   22.6   22.5    -0.1
    1.50   21.0   20.9    -0.1
    1.75   19.6   19.5    -0.1
    2.00   18.3   18.3
    2.25    17.2   17.2
    2.50   16.2   16.1    -0.1
    2.75   15.2   15.2
    3.00    14.4   14.4
    3.25   13.7    13.7
    3.50   13.0   13.0
    3.75    12.3   12.3
    4.00   11.8    11.8
    4.25   11.2   11.2
    4.50    10.7   10.7
    4.75   10.3    10.3
    5.00    9.9     9.9
    6       8.5     8.5
    7       7.4     7.4
    8       6.6     6.6
    9       5.9     5.9
    10      5.3     5.3
    11      4.9     4.9
    12      4.5     4.5
    13      4.1     4.1
    14      3.8     3.8
    15      3.6    3.6
    The NEW  Nautical Almanac refraction table:
    Using a temperature factor of  273.15/283.15 and and changing the lapse rate
    as follows:
    ht<3000: LRate=  -9
    300013000: LRate= 0
    produces a  refraction table very close to standard (post-2004) Nautical
    0.00    33.6   33.8   +0.2
    0.25   30.8    30.9   +0.1
    0.50   28.3    28.3
    0.75   26.1   26.1
    1.00    24.2   24.1   -0.1
    1.25   22.4    22.3   -0.1
    1.50   20.9   20.8    -0.1
    1.75   19.5   19.4    -0.1
    2.00   18.3   18.2    -0.1
    2.25   17.1   17.1
    2.50    16.1   16.1
    2.75   15.2    15.2
    3.00   14.4   14.3   -0.1
    Note that  the differences in these latter tables are on the order of 0.1
    arcMINUTES. Of  course since that's the precision limit of the table, it's hard
    to experiment  with greater accuracy.
    So which of these various lapse rate models is  correct? The short answer is
    'all of them' and 'none of them'. You can find  various diagrams on the net of
    "typical" temperature profiles and "standard"  atmosphere profiles, but all
    of these are just long-term averages and  idealizations. There's a lot of
    variability about what happens in the real world  in the lower atmosphere, and
    there's no particular reason to prefer, for  example, the temperature profile that
    regenerates the current almanac's  refraction table. I really do not believe
    that there is any good reason to  prefer the post-2004 Nautical Almanac
    refraction tables to the pre-2004 tables  (though later evidence may yet persuade
    me!). The difference in the lapse rate  profile does not appear to be
    Is all lost then? Is there  anything positive that comes out of this sort of
    analysis?? I think there is. By  varying the temperature profiles in ways that
    reflect real variability in  Nature, we can determine in an obective (albeit
    theoretical) fashion just how  far we can trust the refraction tables.
    Throwing in temperature inversions,  changes in the lapse rate, the location of the
    lower edge of the stratosphere,  and all of that, I find that the refraction
    tables are 'rock solid' above about  3.0 degrees. Almost nothing that can happen
    in the weather is likely to make any  meaningful change of 0.1 minutes or
    larger above that height. Even at 2.0  degrees altitude, the changes in
    refraction amount to only a few tenths of a  minute. This suggests that refraction
    tables can be relied upon safely down to  very low altitudes (on the other hand,
    under the same conditions that would  yield unusual changes in refraction, the
    'dip' of the horizon will be much more  liable to error).
    I've read in various places before that the refraction  above 10 degrees or
    so is almost completely insensitive to atmospheric  structure, and the
    refraction close to the horizon is sensitive mostly to the  rate of change with height
    of the temperature in the atmosphere. These numerical  integrations confirm
    those general statements beautifully and even place  stronger limits on the
    numbers. Refraction above 15 degrees is COMPLETELY  insensitive to atmospheric
    structure (and for that matter, it can be calculated  easily from k*tan(z)).
    Refraction between 3 and 15 degrees is nearly insensitive  to atmospheric
    structure. And refraction below 3 degrees depends in detail on  the temperature
    structure of the lowest layers of the troposphere. There is no  'correct'
    refraction at such low altitudes. [note that all of this applies to an  observer at sea
    By the way, if anyone would like to experiment with these models, let me
    know. I've been considering making a little web app out of  it.
    42.0N 87.7W, or 41.4N  72.1W.

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site