Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Sensitivity of Mercator sailing to round off error in course
    From: Sam Chan
    Date: 2001 Apr 27, 7:34 PM

    I was looking at the distance calculations from various list members 
    responding to Silicon Sea Leg 70 and was puzzled by the differences in the 
    same calculations. I looked into the equation used for calculating distance 
    and determined that it is very sensitive to round off errors in the course.
    Given a departure and destination point, the equations used to determine 
    course and distance for Mercator sailing is:
    C = arctan (Dlo/DMP), where C is course, Dlo is difference in Longitude in 
    minutes, and DMP is difference in meridional parts.
    D = Dlat cos C, where Dlat is difference in latitude in minutes.
    Typically in writing the course C, it is rounded to 1 decimal place. If this 
    rounded value of C is used in the calculation D, the error can be significant 
    if C is near east or west. The rounding is not an issue if the calculation is 
    being done by a computer program where the full precision of C is maintained.
    The differential of D is dD = D tan C dC. For typical rounding of C to 1 
    decimal place, the error dC is ~+/- 0.05. The resulting error in D, dD, is 
    highly dependent on C. If C is near east or west, tan C is very large. Thus 
    explains the differences in the distances from the various list members for 
    SS 70.
    As an alternate approach for those of us who like to use pencil, paper, and 
    tables to do the calculations, we can first calculate the departure before we 
    calculate D. But we need a calculation of departure not explicitly dependent 
    on course. From Roy Williams, "Geometry of Navigation", we have p = Dlo 
    (Dlat/DMP) where p is the departure. Then distance is then:
    D = sqrt(Dlat^2 + p^2)
    The differential error in this format is dD = (Dlat d(Dlat) + p dp)/D. The 
    error is significantly insensitive to errors in Dlat and p.
    Sam Chan

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site