Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Re: Converting degrees-minutes to degrees and back
    From: Paul Hirose
    Date: 2018 Nov 20, 11:00 -0800

    On 2018-11-18 21:45, Steve E. Bryant wrote:
    > When performing calculations involving degrees and minutes, I much prefer 
    converting the minutes to degrees, proceeding with the additions and 
    subtractions, and then converting the decimal degrees back to minutes.
    A calculator can do sexagesimal addition and subtraction directly with a
    trick from the days of mechanical calculating machines. For example, to
    calculate 1°05′ + 2°10′, enter the operands as 1.05 + 2.10 = 3.15.
    Of course it's not always that simple. A carry may occur, for example in
    1°55′ + 2°30′. The calculator says 3.85. Clearly 85 minutes is illegal.
    To force a carry add 40′ (.4 on the calculator) to obtain 4.25. (Back in
    the day, some computers called this trick the "rule of 40.")
    When the minutes field in the sum is 60 or greater it obviously needs
    adjustment. I call that a "type 1" carry. On the other hand, 50′ + 50′
    comes out as 1.00 on the calculator. That's a "type 2" carry — if you
    look only at the result you don't see anything wrong. However, the type
    2 can be anticipated because it occurs only when the minutes sum is 100
    or greater. That requires a large number of minutes in both operands —
    one in the 50s and the other in the 40s or 50s.
    Degrees (or hours) minutes seconds are no problem. For example, add 1h
    28m 50.4s to 2h 31m 15.8s. Sum on the calculator is 3.59662. Since the
    carry from seconds will force a carry from minutes, you can save time by
    adding 40 to both minutes and seconds with a single operand (.404).
    Result is 4h 00m 06.2s. Note that a carry also occurred from the
    fractional part of seconds, but that's a decimal number so the carry
    took care of itself.
    Subtraction is similar to addition, except that you subtract 40 to force
    a borrow. For instance, work 1°25.2′ - 48.8′ as 1.252 - .488 = .764.
    Subtract 40 from minutes to obtain 36.4′.
    A type 2 borrow occurs when the minutes differ by more than 40. For
    example, 1°01′ - 42′ appears on the calculator as .59. From inspection
    of both operands it's obvious a borrow must occur, so subtract 40′ to
    get the correct 19′.
    The rule of 40 can deal with negative numbers too. By experimentation
    with simple calculations you can work out the rules. With a stack type
    calculator (Hewlett-Packard) it's easy to swap subtraction operands to
    dodge these issues.
    I use the decimal point to separate degrees from minutes, but a
    sexagesimal can be entered as an integer. That's what they did with
    mechanical calculators. A disadvantage is that when an operand is a
    whole number of degrees (as when computing 90 minus something) you have
    to pad the right side with zeros.
    Unfortunately I know no easy way to multiply and divide sexagesimal
    values on a calculator.
    > There may be some statistical reason for not doing it that way; or, is the 
    practice perfectly acceptable.
    > There are those professionals who teach navigation that insist upon working 
    only with degrees and minutes, unless it's necessary to use decimal degrees 
    for calculator entry in which case they carry the decimal out to the fifth 
    I think four places are enough. One minute is about .02°, so a tenth is
    .002°. Thus a conversion to degrees with four decimal places properly
    rounded has far less error than tables rounded to tenth minute precision.

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site