# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Nonlinearity tables, was: Averaging**

**From:**Alexandre Eremenko

**Date:**2004 Oct 23, 16:26 -0500

Here are the detailed non-linearity error tables. The left column is the maximal altitude of the body (at the time of the upper culmination) in degrees. The second column is the lower bound of the (LHA-) distance of the observed body from the meridian at the time of observation, in degrees. Other 2 columns give maximal non-linearity errors under these conditions. (That is by how much the graph of the altitude as a function of time differs from a straight line.) Column 3 is related to the interval of 5 minutes and column 4 to the interval of 4 minutes. (5 observations with 1 minute interval take 4 minutes!) The results are rounded to .1' As I said before, these results were obtained by computing the second derivative of the altitude with respect to time, for abouit 8 million equally spaced points in the parameter space, and then taking maximum over the specified parameter region. MAX. ALT DIST ERROR(5) ERROR(4) 60 0 0.3 0.2 60 20 0.2 0.1 70 0 0.4 0.3 70 20 0.2 0.1 80 0 0.4 0.3 80 10 0.4 0.2 80 20 0.2 0.1 85 0 0.4 0.3 85 10 0.4 0.2 85 20 0.2 0.1 85 45 0.1 0.0 Examples: 1. You made 5 observations with less than 1-minute intervals between them (so the total procedure took no more than 4 min) The body was roughly bearing NE. According to the last row of the table, the non-linearity error is zero. That is when you average the altitudes, you introduce an error which is less than the rounding error of the almanac. 2. You made 5 observations with less than 1-minute intervals between them of the Sun at 10:15 a.m. local time, and your latitude was 30 degrees. Your non-linearity error does not exceed 0.1' according to the second row from below. 3. You averaged 5 observations taken around noon, maybe 2 before noon and 3 after, but it took you no more than 4 minutes to do this. (I don't know why would one do this, but suppose you did) and the altitude was 65. According to row 3 your non-linearity error can be as large as 0.3'. One more simple rule of thumb: If you use a plastic (or other non-metal) sextant averaging the altitudes will ALWAYS increase precision. If you use the "Complete on board Cel Navigator" to reduce your sights averaging will ALWAYS increase your precision. Alex.