# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: R: Problems with AstronavPC**

**From:**George Huxtable

**Date:**2004 Feb 17, 21:35 +0000

Fedirico Rossi wrote about the equations for fix error in AstronavPC- >I've read your email and it proved useful in clarifyng some aspects of >the problem, still one thing is still obscure to me: the formula to >calculate the variance S reads: > >S = F - DdB - EdL cos B(F) > >I guess that D and E are the same that appear in the formulas to >calculate lat and long of fix while B(F) is the latitude of fix. >What about dB and dL? Are they the difference respectively in latitude >and longitude between the fix and the initially assumed position? If so, >what is F? =============== Sorry, I didn't compare the texts, in the Almanac and in AstronavPC, closely enough. Thanks to Federico for seeking clarification. In the Nautical Almanac version, F isn't defined, because it's needed only for the error estimate, which is in the AstronavPC version only. The missing equation is- F = p(1)*p(1) + p(2)*p(2) + ... [the sum of the squares of the intercepts] where brackets are written here, instead of subscripts. As Federico surmised, D and E are the same as in the Almanac section 11. The text of the AstronavPC equivalent of section 11, which is numbered 7.4, is effectively the same as in the Almanac, with the addition of the definition of F, until you get to "where the number of terms in each summation is equal to the number of observations." Then 7.4 continues as follows- ================ "As a check verify that A + C = n where n is the total number of observations included in the solution. Calculate G = A * C - B * B Then an improved estimate of the position at the time of fix L(I) , B(I) is given by L(I) = L(F) + dL and B(I) = B(F) + dB where dL= (AE - BD) / (G cos B(F)) and dB = (CD - BE) / G Calculate the distance d between the initial estimated position L(F) , B(F) at the time of fix and the improved estimated position L(I) , B(I) in nautical miles from- d = 60 * square root of ( (dL * cos B(F))squared + (dB)squared) If d exceeds about 20 nautical miles set L(F) = L(I), B(F) = B(I) and repeat the calculation [note that this involves recalculating the intercepts and azimuths- George] until d, the distance between the previous estimate and the improved estimate, is less than about 20 nautical miles. It is possible but not advisable to start the iterations with a position that is in a different hemisphere. Provided L(I) is kept in the range -180deg to +180deg and B(I) in the range -90deg to +90deg the solution in most cases will begin to converge after a few iterations." =============== end of quote. Some further comments from George- The text above isn't entirely clear to me. As I pointed out earlier, the final paragraph describes choice of starting values for L(I) and B(I), but those are not values that you initially choose, they are the improved long and lat. The chosen starting values are L(F) and B(F). It seems most likely to me that the final paragraph intended to refer to those quantities The size of dL and dB will shrink at every iteration, and if iteration is continued, well past the suggested 20-mile limit then both dL and dB would become arbitrarily small. The equation for S in 7.5 is- S = F - DdB - EdL cos B(F), That calculation for S appears to include an allowance for the "improvement" stage of the last iteration. If reiteration was continued until dL and dB were allowed to shrink to negligible proportions, then we would get convergence toward S = F, where F is the sum of squares of the intercepts. I hope this is somewhat clearer now. Sorry to have complicated matters by omitting vital components. George. ================================================================ contact George Huxtable by email at george@huxtable.u-net.com, by phone at 01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK. ================================================================