NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: Round-off
From: Frank Reed
Date: 2009 May 16, 13:08 -0700
From: Frank Reed
Date: 2009 May 16, 13:08 -0700
I wrote yesterday: "That's right except for one little detail. Round-off error is a random number taken from a uniform sample distribution. What you want for the step size if not the maximum size of the round-off error, but the standard deviation of that. You'll find that this amounts to 1/sqrt(12) times the width of the round-off band. So if you have a dozen numbers added up where you've dropped the tenths, your expected error is only ONE tenth." Sorry, that should have read "ONE UNIT" not "ONE tenth". The standard deviation is 1/sqrt(12). If you have N of 'em, your expected error is sqrt(N) times that standard deviation or in other words sqrt(N/12). -FER --~--~---------~--~----~------------~-------~--~----~ Navigation List archive: www.fer3.com/arc To post, email NavList@fer3.com To , email NavList-@fer3.com -~----------~----~----~----~------~----~------~--~---
