NavList:

A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: Round-off
From: John Karl
Date: 2009 May 18, 20:18 -0700

```Sorry, I messed up my explanation of the expected round off error,

The standard deviation (RMS value) of a uniform random distribution of
width L (center on zero) is L/sqrt(12).  For calculations carried out
to 0.1', L= 0.1'.  For calculations carried out to 1.0', L = 1.0'.

A series of N additions (or subtractions) of these numbers then
produces a standard deviation sqrt(N) greater.  That is, the standard
deviation of the result is now L*sqrt(N/12).  Frank gave this result
also.  This is an interesting result because most of our sight
reductions (lunar maybe excepted) have N less than 12, so we maintain
the precision of the original numbers.  I.e., using all values precise
to 0.1' gives a final result to nearly the same precision of 0.1' --
But remember, that's only one of many different probability
estimates.  The max error is still L*N/2.  And as before, I'm just
talking about round off error here.

JK

--~--~---------~--~----~------------~-------~--~----~
To post, email NavList@fer3.com
To unsubscribe, email NavList-unsubscribe@fer3.com
-~----------~----~----~----~------~----~------~--~---

```
Browse Files

Drop Files

Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

Email Settings

 Posting Code:

Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)