# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: Rounding decimal fractions
From: Stan K
Date: 2013 Jul 20, 20:38 -0400
Herbert,

Is this an example of what you call "double rounding"?

The Power Squadrons has students display azimuth angle, Z, to tenths of degrees, but when it is converted to azimuth, Zn, only to whole degrees.  For example, if Z = N 49.51000º W, this would be displayed as Z = N 49.5º W.  Using this rounded value to calculate Zn you would get Zn = 360º - 49.5º = 310.5º which would round to 311º (USPS rounding).  However, using the unrounded value of Z to calculate Zn to whole degrees you would get Zn = 360º - 49.51000º = 310.49º, which would round to 310º (USPS rounding).

This always bothered me, and when I grade an exam where Z is xxx.5º, I give full credit for either possible answer.

Stan

-----Original Message-----
From: Herbert Prinz <herbert@heavenlysoftware.org>
To: slk1000 <slk1000@aol.com>
Sent: Fri, Jul 19, 2013 1:25 pm
Subject: [NavList] Re: Rounding decimal fractions

Paul,
You say
"I have to admit I never pay attention to these fine points in my own
software. Since Tinyac and Lunar3 both have selectable precision, I
assume the user will use, say, .01 precision if "tenths" are critical."
This may be beneficial in some cases, but it also has the potential of aggravating an error through the effect of double rounding when a user selects .01 precision at first ("to be safe", or "because it does not cost anything") and then finally decides to round the result to 0.1 precision because that is what he really needs.
Herbert Prinz

----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------
View and reply to this message: http://fer3.com/arc/m2.aspx?i=124650
Browse Files

Drop Files

### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

### Email Settings

 Posting Code:

### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)