 NavList:

A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Least squares method for determining a fix
From: Federico Rossi
Date: 2003 Oct 10, 13:39 +0200
Hi,
does anybody know the formulas to calculate the error ellipse around the fix obtained with the least squares method?
I've got some formulas that I found on a Spanish site, but they don't explain what is sigma (I suppose a variance) and most of all how to compute it.
The formulas are (a probability of 0.95 is generally considered):

k=sqrt(-2*log(1-probability)

theta=arctan(2*B/(A-C)

a=sigma*k/sqrt(nObs/2+B/sin(theta))

b=sigma*k/sqrt(nObs/2-B/sin(theta))

where:    a and b are the axis of the ellipse
nObs is the number of sights
A, B and C are parameters calculated with the least squares method as found in the Nautical Almanac

These formulas only give a and b, what about the orientation of the two axis?

Thanks.
Federico Browse Files

Drop Files What is NavList? Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member. Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email: Email Settings

 Posting Code: Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)