# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Online Extended Precision Calculator**

**From:**Harri Ojanen

**Date:**2015 May 4, 04:59 -0700

Hello,

Wolfram Alpha looks quite interesting, but it might not be that mature yet. I'm getting interesting results for this example. It gives the result in radians as

0.3490658503988659153847381536977225426885743777083450912194382880342018229206898887364483139269018964 radians

which is probably correct: it agrees with what Wolfram Mathematica gives for Pi/9, and Mathematica is a mature established product. However, Alpha prints also the following alternative results that it has computed:

19.9999999999999988669706688890675043201776943694917992903726455566788225422415098971607513768758857° (degrees)

and even

19 degrees 59 arc minutes 59.99999999999592109440800064301555263969973017047744534152400404376115206943562977870495675318852 arc seconds

Obviously these other values that it has obtained by converting back to degrees are not correct to the requested number of decimals. Of course, if there is no other way of checking the correctness of the value in radians we wouldn't know if that is correct to 100 decimals or not.

In general regarding the number of decimals that are needed: double precision is usually enough, and also often necessary, even for celestial navigational purposes. You don't need that many decimals for the final result, but the extra digits are sometimes necessary for intermediate results in order for the final result to be accurate to say 7 or 8 digits. If you don't want to worry about when you need the extra precision in intermediate steps, it's easy to use double precision always.

An example is the cosine formula for spherical triangles. For small angles, the accuracy is halved when solving with that formula. So if you used single precision, in some cases the result would be accurate to only about 4 digits, which is not enough even for navigation. You could use the haversine formula instead, but that is inaccurate for some other angles, so basically when working in single precision you need to choose different formulas depending on the values of the angles. In general I would recommend using double precision (i.e., about 16 digits) always, then you don't have to worry about which formula to use (for celestial navigation purposes).

Harri