Welcome to the NavList Message Boards.


A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Compose Your Message

Add Images & Files
    Re: Rounding decimal fractions
    From: UNK
    Date: 2013 Jul 19, 10:02 -0700


    It's exactly the opposite of what you state regarding standard rounding versus banker's . The flaw in your argument that "[...] 'standard' rounding has five values that round down and five that round up, i.e. half down and half up." is that you do not take into consideration the sizes of the individual rounding errors. Simply put, when you "round" 0, you are not really rounding, so don't count it in!

    More correctly argued:

    Consider integers 0 to 9, evenly distributed. The following table shows the signed rounding error when rounding to the nearest 10. (The reason why we choose the NEAREST 10 is that we want to minimize absolute rounding error in each case.)

    0 ... 0
    1 ... -1
    2 ... -2
    3 ... -3
    4 ... -4
    5 ... ?
    6 ... +4
    7 ... +3
    8 ... +2
    9 ... +1

    The arithmetic sum of all cases in the table is zero. Any unbiased rounding algorithms must therefore use a procedure that ensures that the average rounding error for 5 comes out to zero. 'Rounding to even' is the cheapest way of implementing this with reproducibility (= pseudo-randomly).

    Herbert Prinz


    The real problem with the v and d correction table in the N.A. is not at all a rounding problem. This table has a systematic error which I will outline in a separate post. The increments for sun and planets, on the other hand, (and, in all likelihood, also that for Aries and Moon) does have a rounding problem. It uses standard rounding instead of the seemingly more appropriate 'round to even' method. I never inquired why this choice was made.

    NavList message boards and member settings: www.fer3.com/NavList
    Members may optionally receive posts by email.
    To cancel email delivery, send a message to NoMail[at]fer3.com

    Browse Files

    Drop Files


    What is NavList?

    Join NavList

    (please, no nicknames or handles)
    Do you want to receive all group messages by email?
    Yes No

    You can also join by posting. Your first on-topic post automatically makes you a member.

    Posting Code

    Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.

    Email Settings

    Posting Code:

    Custom Index

    Start date: (yyyymm dd)
    End date: (yyyymm dd)

    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site
    Visit this site