 # NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: least squares method for determining a fix
From: Andr?s Ruiz
Date: 2003 Oct 14, 12:44 +0200
Mensaje
Estimated Position Error

If three or more position lines are obtained an estimate of the error in position may be calculated. In general as the number of observation increases the error in the estimated position decreased.

The standard deviation of the estimated position, in nautical miles is:

`S = F-D*dB-E*dL*COS( BI );`
`sigma = 60.0*sqrt( S/(nObs-2.0) );`

The Confidence Ellipse of axes (a,b) is:

`Prob = 0.95;`
`k = sqrt(-2.0*log(1.0-Prob));  // scale factor`
`theta = ATAN( 2.0*B/(A-C) )/2.0;`
`a = sigma * k / sqrt( nObs/2.0 + B/SIN(2.0*theta) );`
`b = sigma * k / sqrt( nObs/2.0 - B/SIN(2.0*theta) );`
-----Mensaje original-----
Enviado el: martes, 14 de octubre de 2003 11:06
Asunto: Re: [NAV-L] least squares method for determining a fix

----- Original Message -----
Sent: Friday, October 10, 2003 1:39 PM
Subject: least squares method for determining a fix

Hi,
does anybody know the formulas to calculate the error ellipse around the fix obtained with the least squares method?
I've got some formulas that I found on a Spanish site, but they don't explain what is sigma (I suppose a variance) and most of all how to compute it.
The formulas are (a probability of 0.95 is generally considered):

k=sqrt(-2*log(1-probability)

theta=arctan(2*B/(A-C)

a=sigma*k/sqrt(nObs/2+B/sin(theta))

b=sigma*k/sqrt(nObs/2-B/sin(theta))

where:    a and b are the axis of the ellipse
nObs is the number of sights
A, B and C are parameters calculated with the least squares method as found in the Nautical Almanac

These formulas only give a and b, what about the orientation of the two axis?

Thanks.
Federico Browse Files

Drop Files ### NavList ### What is NavList? ### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member. ### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email: ### Email Settings

 Posting Code: ### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)