NavList:
A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
Re: GreatCircle Calculations - precision
From: Jared Sherman
Date: 2002 Oct 2, 18:05 -0400
From: Jared Sherman
Date: 2002 Oct 2, 18:05 -0400
Rino- I'm no mathematician, and won't even play one on TV.Obvious first thought is whether you are seeing rounding errors or other math errors caused by the way your programming language is doing the math? I remember hearing about single/vs/double precision integer math and other limits and problems from long ago, I suspect those problems still exist to some extent in all programming languages or operating systems. At least, that's a first area to eliminate.
