A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
From: Frank Reed
Date: 2017 Jun 28, 11:52 -0700
Andrés Ruiz, you wrote:
"So, I took an example from a book, and did the inverse calculating; the true altitudes from the position, and then, I introduced a systematic error in altitudes."
Well, it was a nice try. But this is a misleading approach to an interesting issue -- specifically, how does a celestial navigator detect and deal with systematic error? By posting your puzzle with 8 digits past the decimal point in the altitudes, you provided no clue that you might be interested in a real-world navigational issue and instead created the impression that you were looking for some sort of subtle mathematical or algorithmic issue ... as David Pike noted, he deduced that this puzzle was, by design, a "battle of the apps". I read your puzzle and didn't even bother to work it up since I saw no evidence in the setup of the puzzle that you were looking at a real navigation issue.
You then asked:
"With this clue, what is the fix?"
The fix is where it was previously calculated. Unless you have additional navigational information (for example, a DR position in which you have considerable known confidence, or double sights of each body, or prior knowledge that you omitted some fixed offset correction, like dip), there is no legitimate excuse to introduce a systematic error correction. This technique of closing the triangle is seductive, dangerous and in fact damn stupid in most cases where it has been historically applied. Unfortunately it became popular lore on USN vessels forty or fifty years ago and from there diffused into wider practice. It was a key component in the three-body fix ritual for some fastidious navigators in the 1970s. Navigators were taught by other navigators that this was a "seat of the pants" method that they "don't teach in those darn textbooks". Some navigators were trained to do this on each and every three-body fix, but it's rotten practice --more late-20th century entropy mucking up the works. A systematic error should never be assumed or calculated unless there is direct, positive reason to believe that such an error exists, or unless more than three sights are available.
"With this example I want to show that the most robust methods have their weaknesses."
Your example as given does not really do that. You could, of course, create an example like that by including some information that provides a real-world, positive reason for introducing a systematic error. You could say "I left out dip" or you could say "we have high confidence that we are within five miles of our DR and the sextant I used is suspect". Also, you might consider listing the observed altitudes to two or at most three digits past the decimal point to avoid creating the impression that you're posing a pure math problem.
By the way, I would emphasize that in cases where there are more that three sights on more than two azimuths (I think that's enough conditions but I haven't thought it through completely), it is then mathematically legitimate to calculate a systematic error in every case. You can think of it as "solving for index error as an unknown". With enough sights, this is possible.