Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Single precision sight reduction
    From: Paul Hirose
    Date: 2015 May 07, 15:07 -0700

    Recently there has been discussion of sight reduction accuracy with
    single precision floating point values. On the principle that an ounce
    of experimental data is worth a pound of theory, I ran a Monte Carlo
    simulation. In my C# source code the formulas are:
    Hc = (float)Math.Asin(sinL * sind + cosL * cosd * cosLHA);
    Z = (float)Math.Atan((cosd * sinT) /
                 (cosL * sind - sinL * cosd * cosT));
    if (Z < 0.0)
        Z = (float) (Z + Math.PI);
    The (float) keyword forces a value into single precision. It's necessary
    because the arc sine and arc tangent functions return a double precision
    result. All variables are single precision.
    The test for negative Z is necessary because the Atan() function returns
    a result in the interval [+pi/2, -pi/2] radians, i.e., [+90, -90]
    degrees. So if it's negative, the angle is converted into the equivalent
    in the range 90 to 180 degrees.
    To generate a sight reduction problem, the program randomly selects a
    latitude, azimuth, and altitude. Latitude and altitude can be restricted
    to less than the full 0 and 90 degree range if desired, but I didn't do
    so in this test.
    With simple random selection of altitude, half the stars will be above
    45°. But that's less than half the area of the visible hemisphere. So
    altitude selection is weighted to generate constant star density
    throughout the sky.
     From latitude, azimuth, and altitude the program computes LHA and
    declination. These data are passed to the sight solver routine. Up to
    this point all computations are in double precision. The solver computes
    altitude and azimuth with the desired reduction method. Its results are
    compared to the correct answers, and the errors recorded.
    For this test I used 100 000 simulated observations. Root mean square
    error in azimuth and altitude was .000667′ and .000296′ respectively.
    Max azimuth error was .0878′ at altitude 89°40.2′. Max altitude error
    was .0132′ at altitude 89°40.9′.
    In production code I wouldn't use the above formulas. The one for Hc
    loses accuracy the higher the body, and the one for Z fails if Z = 90°
    since the tangent of that angle is undefined. Trouble isn't likely with
    real world data, but a user experimenting with the program can create a
    degenerate case. You can blow up the Z formula with zero latitude and

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site