A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding
From: David Iwancio
Date: 2022 May 11, 17:49 -0700
Something to keep in mind when applying regression algorithms to data is keeping in mind what the algorithm believes to be unknown rather than known.
For example, with linear regression where you're computing both "m" and "b" in the formula "y = m(x) + b," you're implicitly assuming you know neither "m" nor "b." But with lunar distances you are relying on a known "m," specifically the idea that the distance changes at a fixed rate of e.g. +31.4' per hour from 06h to 07h. If you're confident that you know UT to the nearest hour, the only truly unknown constant is "b."
Using the known rate-of-change of the lunar distance and the time elapsed between sights in a single sighting session, you can reduce the sights to a common point in time and then analyze the results afterwards (e.g. an arithmetic mean to discern between random and fixed errors). It's the same concept of advancing/retiring a line of position as you move during a single sighting session (where your knowledge of course and speed is much more reliable than your sextant readings), or the fit-slope method mentioned in section 1805 of the current edition of Bowditch vol. 1.
Something else you might consider is assigning statistical weights to your sights. Before looking at the vernier and seeing if the reading is where you think it "should" be, take a note of how confident you feel in a particular sight, where you could later assign some numerical value to your confidence level. You could also gradually reduce the weight of later sights as your fatigue grows over time.
As an aside, you could also assign weights to different lines of position based on what an error of 0.1' would make. An LoP from the altitude of a star (0.1' = 0.1 nmi) should have a much greater weight than longitude by lunar distance (0.1' = ~20 nmi), but much less weight than a distance from a landmark over the horizon (0.1' = ~0.01 nmi).