# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

Message:αβγ
Message:abc
 Add Images & Files Posting Code: Name: Email:
Re: Round-off
From: Jeremy C
Date: 2009 May 17, 18:40 -0700

```Peter: Point taken, but I'd still try to keep that extra decimal point of data
since it really shouldn't add any significant time in standard reduction
methods (like HO 229).  What do the power squadron people teach in this
regard, since they are generally teaching the small boat crowd?

Lu Wrote:

On May 17, 8:28 pm, Lu Abel  wrote:
> In most engineering schools students are taught to carry out
> calculations to at least one more digit of accuracy that is desired in
> the final answer and only round at the very end.

I have met very few mates who are engineers.  In fact I am one of the few.
Most hate math in general.  It is a great rule however.  I like using all the
precision I have availabe with my sources which typically is 0.1 (sextant,
Nautical Almanac, and books)
>
> I also have to wonder about "0.1' of arc and 0.1 deg of azimuth"   0.1
> degree of azimuth is exceedingly hard to measure either with a compass
> or to plot on a chart, whereas 0.1' of arc is 0.1 nm, easily discernible
> on most mid- to large-scale charts or on a plotting sheet used to reduce
> two LOPs to a L/Lo of position.
>

I think we need to clarify what we are using.  For plotting sheets I am using
either a universal plotting sheet which is covering several degrees of
latitude and longitude in a fairly small piece of paper.  I can't plot to 0.1
for lat or long on this with any confidence at all.  0.3-0.5' is a good
guess.  On the ship I am using PS92x series ploting sheets for specific
latitudes which are much larger and are a bit better scale.  I would be
confident at 0.2 for lat and long on this sheet.  I certainly can't
accurately draw 10 lines between each minute of lat or long on either of
these charts so will not claim 0.1 accuracy.

I use navigation triangles for my azimuth mesurements on the charts and can
draw much less than 1 deg with regularity.  For measuring azimuths with my
Alidade (magnified) I can observe bearings with an accuracy of 0.2 deg (esp
stars), but with the azimuth circle its 0.5 deg at best.  When making compass
correction via triple interpolation of HO 229 I look for 0.1 deg in my
reduction with my biggest error being found in the references of variation
when out on the open ocean.  In reality I am usually getting 0.3-0.4 degrees
of accuracy in plotting but will certainly go to 0.1 with the math.

If you are using charts with 0.2' hash marks I can see the precision that you
speak of, but I can't think of any time that I've plotted Celnav LOP's on
such a large scale chart.  I am using piloting techniques at that point.

I guess my question is what plotting sheets are you using that you can
deterimine Lat/Long to 0.1'?  I find that my 0.5mm pencil lead is at least
0.2' wide on thse sheets, and on the universal plotting sheets is closing in
on 0.5' wide.

I am attaching a pdf reference that shows a plot on one of the big Plotting
sheets I use as well as my triangles.

Jeremy
File:

--~--~---------~--~----~------------~-------~--~----~
To post, email NavList@fer3.com
To unsubscribe, email NavList-unsubscribe@fer3.com
-~----------~----~----~----~------~----~------~--~---

```
Browse Files

Drop Files

### Join NavList

 Name: (please, no nicknames or handles) Email:
 Do you want to receive all group messages by email? Yes No
You can also join by posting. Your first on-topic post automatically makes you a member.

### Posting Code

Enter the email address associated with your NavList messages. Your posting code will be emailed to you immediately.
 Email:

### Email Settings

 Posting Code:

### Custom Index

 Subject: Author: Start date: (yyyymm dd) End date: (yyyymm dd)