NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: Rounding decimal fractions
From: Stan K
Date: 2013 Jul 20, 18:43 -0400
From: Stan K
Date: 2013 Jul 20, 18:43 -0400
Herbert,
Just got back from attempting to walk Battle Road at Minute Man National Historical Park in Concord, Massachusetts yesterday, with temperatures of 99-100ºF. Didn't make it
. Gave up pretty early on.
Anyway, it seems that I inadvertently started a fairly interesting thread about rounding. The only reason I ever mentioned rounding was as a lead-in to how I accidentally found the differences between the corrections values in the Nautical Almanac I&C tables before and after 2002 (or 2001?).
First, I should never have called the 0-4 round down, 5-9 round up scheme "standard" rounding. I should have called it United States Power Squadrons rounding, as this is the way, right or wrong, that it is taught in all USPS courses. Celestial Tools was intended to be a learning aid for USPS courses rather than a practical, on-the-water tool, and, as such, it had to reproduce a student's work as closely as possible. One of the places I thought that would be easy was the Increments and Corrections tables. Just use the data provided in the Explanations at the back of the Nautical Almanac (and elsewhere) and round to one decimal place. Using USPS rounding did not work, and I don't recall if I tried any other method. (And, according to your P.S. it would not matter what kind of rounding I used for the v and d corrections anyway). Nevertheless, I believe I managed to reproduce all ~11000 increments and all but seven of the ~11000 corrections with "Stan" rounding, and hard-coded those seven into the program, so all ~22000 match. Well, at least they match the "modern" Almanacs.
Incidentally, I also checked NavSoft's I&C tables and found that neither the increments nor the corrections match the printed Nautical Almanac perfectly, so I contacted them. They use the same data with some kind of "normally accepted" rounding scheme (not sure which), resulting in values that are often 0.1' different than those of the printed Almanac. I also looked at the French Ephemerides Nautiques. I don't recall how the increments compared, but since it only shows corrections for every 0.3' change in v or d (rather than every 0.1') I didn't bother with it.
But all that is unimportant compared to the fact that four of the corrections in the Almanac we commonly use really did change, so those of us who have been teaching that the I&C tables from any Almanac can be used for USPS exams and homework problems have to provide a caveat. However, it should be easy to create problems that avoid those four values.
Looking forward to reading your next message regarding your P.S.
Stan
Just got back from attempting to walk Battle Road at Minute Man National Historical Park in Concord, Massachusetts yesterday, with temperatures of 99-100ºF. Didn't make it
. Gave up pretty early on.Anyway, it seems that I inadvertently started a fairly interesting thread about rounding. The only reason I ever mentioned rounding was as a lead-in to how I accidentally found the differences between the corrections values in the Nautical Almanac I&C tables before and after 2002 (or 2001?).
First, I should never have called the 0-4 round down, 5-9 round up scheme "standard" rounding. I should have called it United States Power Squadrons rounding, as this is the way, right or wrong, that it is taught in all USPS courses. Celestial Tools was intended to be a learning aid for USPS courses rather than a practical, on-the-water tool, and, as such, it had to reproduce a student's work as closely as possible. One of the places I thought that would be easy was the Increments and Corrections tables. Just use the data provided in the Explanations at the back of the Nautical Almanac (and elsewhere) and round to one decimal place. Using USPS rounding did not work, and I don't recall if I tried any other method. (And, according to your P.S. it would not matter what kind of rounding I used for the v and d corrections anyway). Nevertheless, I believe I managed to reproduce all ~11000 increments and all but seven of the ~11000 corrections with "Stan" rounding, and hard-coded those seven into the program, so all ~22000 match. Well, at least they match the "modern" Almanacs.
Incidentally, I also checked NavSoft's I&C tables and found that neither the increments nor the corrections match the printed Nautical Almanac perfectly, so I contacted them. They use the same data with some kind of "normally accepted" rounding scheme (not sure which), resulting in values that are often 0.1' different than those of the printed Almanac. I also looked at the French Ephemerides Nautiques. I don't recall how the increments compared, but since it only shows corrections for every 0.3' change in v or d (rather than every 0.1') I didn't bother with it.
But all that is unimportant compared to the fact that four of the corrections in the Almanac we commonly use really did change, so those of us who have been teaching that the I&C tables from any Almanac can be used for USPS exams and homework problems have to provide a caveat. However, it should be easy to create problems that avoid those four values.
Looking forward to reading your next message regarding your P.S.
Stan
-----Original Message-----
From: Herbert Prinz <herbert@heavenlysoftware.org>
To: slk1000 <slk1000@aol.com>
Sent: Fri, Jul 19, 2013 1:06 pm
Subject: [NavList] Re: Rounding decimal fractions
----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------
From: Herbert Prinz <herbert@heavenlysoftware.org>
To: slk1000 <slk1000@aol.com>
Sent: Fri, Jul 19, 2013 1:06 pm
Subject: [NavList] Re: Rounding decimal fractions
Stan,
It's exactly the opposite of what you state regarding standard rounding versus banker's . The flaw in your argument that "[...] 'standard' rounding has five values that round down and five that round up, i.e. half down and half up." is that you do not take into consideration the sizes of the individual rounding errors. Simply put, when you "round" 0, you are not really rounding, so don't count it in!
More correctly argued:
Consider integers 0 to 9, evenly distributed. The following table shows the signed rounding error when rounding to the nearest 10. (The reason why we choose the NEAREST 10 is that we want to minimize absolute rounding error in each case.)
0 ... 0
1 ... -1
2 ... -2
3 ... -3
4 ... -4
5 ... ?
6 ... +4
7 ... +3
8 ... +2
9 ... +1
1 ... -1
2 ... -2
3 ... -3
4 ... -4
5 ... ?
6 ... +4
7 ... +3
8 ... +2
9 ... +1
The arithmetic sum of all cases in the table is zero. Any unbiased rounding algorithms must therefore use a procedure that ensures that the average rounding error for 5 comes out to zero. 'Rounding to even' is the cheapest way of implementing this with reproducibility (= pseudo-randomly).
Herbert Prinz
P.S.
The real problem with the v and d correction table in the N.A. is not at all a rounding problem. This table has a systematic error which I will outline in a separate post. The increments for sun and planets, on the other hand, (and, in all likelihood, also that for Aries and Moon) does have a rounding problem. It uses standard rounding instead of the seemingly more appropriate 'round to even' method. I never inquired why this choice was made.
----------------------------------------------------------------
NavList message boards and member settings: www.fer3.com/NavList
Members may optionally receive posts by email.
To cancel email delivery, send a message to NoMail[at]fer3.com
----------------------------------------------------------------
