# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: Haversine formulae for Great Circles
From: Geoff Kuenning
Date: 2001 Nov 25, 11:30 PM

```Lu Abel writes:

> As a computer programmer myself, I'd rather have ONE formula that worked
> for all distances than have to go through some complex logic to switch from
> one formula to another.  Plus any time one does a switch there's always a
> chance the answer will jump, and for sure some customer would call and
> complain "I moved 10 ft and the DTG jumped 100 ft, your software must have
> a bug in it!"

Although it would be nice to have a single formula, it's quite common
in programming to be forced to use two or more.  As several people
have already pointed out, the problem is that computers necessarily
have limited precision.  As it turns out, when dealing with fractional
numbers the worst errors occur during addition and subtraction.  Many
serious scientific applications deal with these sorts of errors by
choosing the best formula for the situation.

The real problem is that most computer science curricula don't cover
issues of numerical accuracy, so there are hundreds of thousands of
programmers out there who have at best a dim grasp of the problem.
Some of them are writing GPS software.

As far as having the DTG jump 100 feet as a result of a 10-foot move,
the hypothetical software *does* have a bug in it.  Switches between
formulas should always be done at a place where the difference in the
answer is minor.  For example, that 100-foot jump in DTG isn't going
to matter much (or even be noticeable) if the distance is 5000 miles.
Alternatively, the two formulas could be "blended" for a while so that
the change appears smooth, although that approach can cause other
problems.

Off-topic, the worst example of the "jump" phenomenon that I know of
is on the F-16 fighter jet.  When you get within a few hundred feet of
the ground in takeoff or landing mode, the responsiveness of the stick
suddenly changes.  Imagine if you were taking a high-speed offramp
and the angle of the front wheels suddenly changed without your having
moved the steering wheel!  (To be fair to the F-16 designers, there
may be control-system issues, which I won't go into here, that forced
them into this horrible design.)
--
Geoff Kuenning   geoff@cs.hmc.edu   http://www.cs.hmc.edu/~geoff/

Orchestra retrospectively extremely satisfied with symphony [No. 1] as
result of barrel of free beer.
-- Gustav Mahler, post-premiere letter to Arnold Berliner
```
Browse Files

Drop Files

### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

### Email Settings

 Posting Code:

### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)