Welcome to the NavList Message Boards.

NavList:

A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Message:αβγ
Message:abc
Add Images & Files
    or...
       
    Reply
    Re: Refraction, Cassini model
    From: Paul Hirose
    Date: 2018 Nov 27, 14:04 -0800

    I added another version of the Cassini model so now there are three to
    evaluate. In addition, my refraction tables have been expanded to
    1-degree steps from 10° to the horizon. To save readers the bother of
    referring to my original message, I'll restate the formulas here and
    include some new material.
    
    
    CORRECTIONS FOR TEMPERATURE AND PRESSURE
    
    For standard sea level conditions I assume 50% humidity, 10° Celsius (50
    F), and 1010 millibars (29.83 inches Hg). For different temperature or
    pressure, compute correction factor f. If P is pressure (millibars) and
    T is temperature (Celsius),
    
    f = .280 * P / (273 + T)
    
    For inches Hg and Fahrenheit,
    
    f = 17.1 * P / (460 + T)
    
    In those formulas P is the actual air pressure at the observer. It can
    be calculated from altimeter setting A for an observer h meters above
    sea level:
    
    P = A * (1 - h / 44 300) ^ 5.26
    
    For h in feet,
    
    P = A * (1 - h / 145 000) ^ 5.26
    
    In both formulas A can be millibars or inches of mercury. The result
    will have the same unit of measure. For example, at 5000 feet and
    altimeter setting 29.8 inches, P = 24.78 inches. A precise calculation
    says 24.80.
    
    I use the term "altimeter setting" instead of "barometric pressure"
    because when the latter is publicly distributed (in the US at least)
    it's really the former. Unless best accuracy is important, it's enough
    to use a fixed nominal altimeter setting since the normal range is
    small. A 2% change, say from 29.9 to 30.5 inches, is unusual. A
    corresponding change in absolute temperature, say from 60 to 70 F, is
    easily exceeded every day.
    
    
    REFRACTION MODELS
    
    "Refco" is the routine from the International Astronomical Union SOFA
    (Standards of Fundamental Astronomy) library. Its documentation says its
    accuracy is better than 1″ at 10° altitude, compared to a ray trace
    through a model atmosphere. Performance below 10° is not stated.
    
    http://www.iausofa.org/2018_0130_C/sofa/refco.html
    
    
    "Low alt" is the formula in the Astronomical Almanac section B for
    altitude less than 15°. I have factored its polynomials into a format
    convenient for a computer program:
    
    refraction = P * (.1594 + (.0196 + 2e-5 * a) * a) / (273 + T) / (1.0 +
    (.505 + .0845 * a) * a))
    
    where "a" is altitude (degrees) and the result is degrees. At the
    horizon in standard conditions it gives a reasonable 34.1′.
    
    
    Cassini's refraction model is explained by astronomer Bruce Young:
    
    https://aty.sdsu.edu/explain/atmos_refr/understanding.html
    
    In standard conditions:
    
    R = Earth radius = 6,371,000 meters
    h = height of homogeneous atmosphere = 8307 m
    n = air index of refraction = 1.0002816
    Z = zenith distance = 90° - altitude
    
    First compute a quantity I call "sinZ1":
    
    sinZ1 = R / (R + h) * sin Z
    sinZ1 = .9987 * sin Z
    
    Then compute refraction from sinZ1 and the index of refraction:
    
    refraction = arcsin (n * sinZ1) - arcsin (sinZ1)
    
    To be strictly correct, R is the distance from the observer to Earth
    center and therefore should include height above sea level. However, at
    .01′ accuracy and 2000 meters (6600 feet) above sea level the small
    adjustment has no noticeable effect.
    
    In the tables I evaluate three Cassini versions.
    
    "Cas1" is refraction from the basic formula, multiplied by correction
    factor f. In other words, Cas1 assumes refraction is directly
    proportional to air density.
    
    "Cas2" applies separate corrections to homogeneous atmosphere height h
    and refractive index n. It assumes refractivity (the fractional part of
    index of refraction) is proportional to air density:
    
    h = 8.22 * P / f  (P is millibars)
    h = 279 * P / f   (P is inches Hg)
    n = 1.0 + f * .0002816
    
    "Cas3" is a formally correct implementation of the Cassini model. Height
    of observer is applied to R, h is computed as in Cas2, and refractive
    index n is computed from temperature, pressure, and humidity with the
    Ciddor formulas, which are too long to quote. See this NIST page:
    
    https://emtoolbox.nist.gov/Wavelength/Documentation.asp
    
    
    "Ben1" is the basic Bennett formula in the Nautical Almanac:
    
    R = 1 / tan (H + 7.32 / (H + 4.32))
    
    where H is refracted altitude (degrees) and R is refraction (minutes).
    
    "Ben2" adds the correction from "Astronomical Algorithms" by Meeus:
    
    -0.06′ * sin (14.7 * R + 13)
    
    where R is the refraction (minutes) from the basic Bennett formula but —
    CAUTION — the result of the expression in parentheses is degrees. The
    mixture of degrees, minutes, and radians requires careful coding on a
    computer.
    
    
    RESULTS
    
    At 10° and up, the SOFA Refco routine is claimed to be highly accurate.
    Therefore, it is the standard, and refraction values from the other
    models are the difference from Refco. Below 10° I use the Astronomical
    Almanac low altitude formula as the reference.
    
    The first table is at sea level standard conditions of 10 C (50 F), 1010
    millibars (29.83 inches Hg) and 50% humidity. Altitude is the refracted
    altitude.
    
            Refco   low alt    Cas1     Cas2     Cas3     Ben1     Ben2
      45     0.97     0.22     0.00     0.00     0.00     0.03     0.00
      40     1.15     0.18     0.00     0.00     0.00     0.03     0.00
      35     1.38     0.15     0.00     0.00     0.00     0.04     0.01
      30     1.67     0.11     0.00     0.00     0.00     0.05     0.01
      25     2.06     0.07     0.00     0.00     0.00     0.06     0.01
      20     2.63     0.03     0.00     0.00     0.00     0.07     0.02
      15     3.55     0.00     0.00     0.00     0.00     0.08     0.03
      10     5.29    -0.01     0.00     0.00     0.00     0.10     0.04
    
       9     0.00     5.83     0.00     0.00     0.01     0.11     0.05
       8    -0.02     6.51     0.00     0.00     0.00     0.11     0.05
       7    -0.05     7.35    -0.02    -0.02    -0.01     0.10     0.05
       6    -0.14     8.41    -0.05    -0.05    -0.05     0.09     0.05
       5    -0.36     9.80    -0.12    -0.12    -0.11     0.07     0.06
       4    -1.00    11.66    -0.28    -0.28    -0.28     0.05     0.07
       3    -3.35    14.28    -0.70    -0.70    -0.69     0.02     0.10
       2   -15.86    18.12    -1.87    -1.87    -1.86     0.00     0.14
       1  -172.34    24.12    -5.24    -5.24    -5.22    -0.02     0.18
       0      ---    34.13   -14.05   -14.05   -14.03    -0.33     0.30
    
    Cassini is practically a perfect match to the reference models — better
    than either Bennett formula — down to 7°. (In standard atmosphere
    conditions the three Cassini versions should be identical, but tiny
    differences occur due to my rounded constants.) At very low altitude
    Bennett is remarkably close to the low altitude formula. However, we
    must remember the latter is only another model and not the absolute
    truth. Accuracy of a few hundredths of a minute in this part of the
    table is an illusion.
    
    
    In the second table air density is 116% of standard: pressure 1070 mb
    (31.60 inches) and temperature -15 C (5 F).
    
            Refco   low alt    Cas1     Cas2     Cas3     Ben1     Ben2
      45     1.12     0.26     0.00     0.00     0.00     0.03     0.00
      40     1.34     0.21     0.00     0.00     0.00     0.04     0.00
      35     1.60     0.17     0.00     0.00     0.00     0.05     0.01
      30     1.94     0.12     0.00     0.00     0.00     0.05     0.01
      25     2.40     0.08    -0.01     0.00     0.00     0.06     0.02
      20     3.07     0.03    -0.01    -0.01     0.00     0.07     0.02
      15     4.14    -0.01    -0.02    -0.01     0.00     0.09     0.03
      10     6.18    -0.04    -0.03    -0.01     0.00     0.09     0.03
    
       9     0.04     6.78     0.00     0.04     0.05     0.13     0.07
       8     0.04     7.56     0.00     0.04     0.06     0.13     0.08
       7     0.03     8.54    -0.02     0.05     0.06     0.12     0.08
       6    -0.03     9.77    -0.06     0.04     0.06     0.11     0.09
       5    -0.18    11.38    -0.14     0.02     0.04     0.09     0.09
       4    -0.71    13.55    -0.33    -0.07    -0.04     0.06     0.11
       3    -2.83    16.59    -0.81    -0.36    -0.32     0.03     0.13
       2   -14.88    21.06    -2.18    -1.39    -1.33     0.00     0.13
       1  -171.99    28.03    -6.09    -4.82    -4.75    -0.02     0.17
       0      ---    39.66   -16.32   -14.77   -14.69    -0.38     0.42
    
    Down to 10° Cas3 is a perfect match to Refco, Cas2 practically as good,
    and Cas1 only slightly less so. All three are better than Ben1 and Ben2,
    though any advantage over the latter is marginal. At 9° the change of
    reference model accounts for the discontinuity in the table. The
    simplest Cassini version, Cas1, begins with the lead, but soon loses to
    Cas2. The complex refractive index computation in Cas3 is no advantage here.
    
    
    In the third table air density is 84% of standard. Pressure is 896 mb
    (29.46 inches), equivalent to 1000 meters (3300 feet) above sea level
    and temperature 25 C (77 F).
    
            Refco   low alt    Cas1     Cas2     Cas3     Ben1     Ben2
      45     0.81     0.19     0.00     0.00     0.00     0.03     0.00
      40     0.97     0.16     0.00     0.00     0.00     0.03     0.00
      35     1.16     0.13     0.00     0.00     0.00     0.04     0.01
      30     1.40     0.09     0.00     0.00     0.00     0.04     0.01
      25     1.73     0.06     0.00     0.00     0.00     0.05     0.01
      20     2.21     0.03     0.00     0.00     0.00     0.06     0.02
      15     2.98     0.01     0.01     0.00     0.00     0.08     0.03
      10     4.43     0.01     0.02     0.01     0.00     0.11     0.05
    
       9    -0.03     4.91     0.00    -0.01    -0.02     0.09     0.03
       8    -0.05     5.48     0.00    -0.03    -0.03     0.09     0.03
       7    -0.10     6.19    -0.02    -0.05    -0.06     0.09     0.03
       6    -0.19     7.08    -0.04    -0.09    -0.10     0.08     0.03
       5    -0.43     8.25    -0.10    -0.18    -0.19     0.06     0.03
       4    -1.07     9.82    -0.24    -0.36    -0.38     0.04     0.03
       3    -3.34    12.03    -0.59    -0.79    -0.82     0.02     0.06
       2   -15.07    15.26    -1.58    -1.92    -1.95     0.00     0.12
       1  -158.61    20.32    -4.41    -4.95    -4.99    -0.01     0.21
       0      ---    28.75   -11.83   -12.47   -12.53    -0.28     0.21
    
    Cas1 is practically as good as its siblings down to 10°. Below 10° the
    simplest Cassini version is best and the most complex is worst! But all
    are within 2% of the low altitude formula down to 5°.
    
    Based on my results, I don't think the complexity of a precise
    refractive index calculation in the Cas3 model is worthwhile. The
    difference from the simple corrections of Cas1 and Cas2 are
    insignificant for navigation. It's difficult a choose the winner between
    the latter two, so I suppose it should be Cas1 due to its simplicity.
    Since it applies the air density correction to refraction itself, the
    expression R / (R + h) in the formula for sinZ1 is a constant.
    
    All three Cassini versions have superior accuracy to even the improved
    Bennett model except at very low altitude where no formula is reliable.
    However, the Cassini model in any variant requires evaluation of three
    trig functions, vs. none in Ben1 and one in Ben2.
    
    Though it's more than 300 years old, Cassini's refraction model with
    simple corrections for nonstandard conditions has amazing accuracy —
    more than enough for celestial navigation.
    

       
    Reply
    Browse Files

    Drop Files

    NavList

    What is NavList?

    Join NavList

    Name:
    (please, no nicknames or handles)
    Email:
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
    Email:

    Email Settings

    Posting Code:

    Custom Index

    Subject:
    Author:
    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site