# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: UTM to lat/lon formulas**

**From:**George Huxtable

**Date:**2003 Dec 13, 11:49 +0000

Thanks to both Paul Hirose and Herbert Prinz for their investigations into problems with the conversion algorithms between lat/lon and UTM, in Seidelmann's "Explanatory supplement to the Astronomical Ephemeris, chapter 4. My own copy appears to be first-edition. At least, there's no mention of any new edition or revision, just a copyright date of 1992. And yet, there seems to be a discordance with Herbert's copy. Herbet said- >Special thanks to Paul Hirose for pointing out an error in the Expl. >Suppl. After >perusing chapter 4, I find that the problem is actually deeper than a mere >printing error. In fact, there is a general mix-up of the various curvatures of >the suface of the spheroid. This may not only affect conversion of geodetic >lat/lon to UTM coordinates, but also datum conversions, as well as the >computation of ECIF positions as required for finding topocentric places of >celestial objects. > >Already on page 206, in formula 4.22-9, we find the correct formula > > N(phi) = a / Sqrt(1- Sqr(e) * Sqr(sin(phi))) > >The magnitude N(phi) is the radius of curvature in the plane perpendicular >to the >meridian. It is required for the computation of geocentric cartesian >coordinates >from the geodetic spherical ones, such as in formula 4.22-7. The only >problem is >that the explanation in the line preceding formula 4.22-9 says that N(phi) >is the >curvature in the meridian, which is wrong. A reader who ignores that >explanation >and does not care what N(phi) means can safely use 4.22-7 in connection with >4.22-9 and get the right results. So far, so good (or bad): my own copy is exactly the same, with the same mis-description of N(phi), as the radius of curvature IN the meridian, though Herbert informs us that it is really the radius of curvature in the plane AT RIGHT ANGLES TO the meridian. >Later, on page 210, a formula for the curvature in the meridian is given again. >This time it is stated as > > N(phi) = a * (1-Sqr(e)) / (1- Sqr(e) * Sqr(sin(phi))) ^ (3/2) > >The second member is indeed the correct term for the curvature of the >meridian at >the given latitude, but it should of course not be named N(phi), lest some >confusion is prone to arise. My own copy is different, on page 210. In mine, the author states, after defining various eccentricities- "N(phi) = ellipsoidal radius of curvature in the prime vertical". The prime vertical is the plane at right angles to the meridian, so now N(phi) is defined correctly here (and different to the written definition on page 206). In brackets that definition is followed by (in Herbert's notation)- N(phi) = a / Sqrt(1- Sqr(e) * Sqr(sin(phi))), exactly the same as on page 206. (Take care with this notation, however. I recall that some (old?) languages use sqr to mean "square-root-of". Herbert means "square-of", and "sqrt" means "square-root-of".) So this would seem now to be completely correct, in definition and equation, and should give the right value for N(phi) to plug into the equations on page 212-213. So for those that have the same version of page 210 as I do, can we ignore Herbert's advice as follows? =================== >Let's call it R(phi), in the following. So, we adopt > > R(phi) = a * (1-Sqr(e)) / (1- Sqr(e) * Sqr(sin(phi))) ^ (3/2) > >Now, it turns out that R(phi) is never needed for the conversion between >lat/lon >and UTM. Whenever the supplement refers to N(phi) in chapter 4.233, equation >4.22-9 is the correct one to use and page 210 is to be ignored, along with the >explanation accompanying 4.22-9. ============================== Herbert goes on to say- >But R(phi1), being introduced in formula >4.233-14 appears exclusively in 4.233-8 in the term N(phi1)/R(phi1), which can >obviously be reduced to > > NoverR(phi1) = (1-Sqr(e)*Sqr(sin(phi1))) / (1-Sqr(e)) > >The same substitution can also be made in the equations given on the web >page the >reference to which I gave earlier. Yes, that's a worthwhile simplification. In my version of the book, there's no mention or definition of R(phi), or of the radius of curvature IN the plane of the meridian, except for that erroneous definition back on page 206 (eq. 4.22-9). R appears without explanation, in eq 4.233-14, as R1 in the special case for angle phi1 as- R1 = a * (1-Sqr(e)) / (1- Sqr(e) * Sqr(sin(phi1))) ^ (3/2), but nowhere in my copy can I find any expression dealing with the general-case of radius of curvature IN the plane of the meridian, at a latitude phi, with its right-hand side equivalent to that quoted by Herbert as- R(phi) = a * (1-Sqr(e)) / (1- Sqr(e) * Sqr(sin(phi))) ^ (3/2) To sum up, if there are no other errors than those described by Herbert Prinz, then it appears that I should be able to use the equations in my copy of Seidelmann as they stand, taking N(phi) from either page 4.22-9 on page 206 or from the identical equation on page 210. The only remaining error is that definition of N(phi) on page 206. Does that make sense to Herbert? If there has been a change in the text, it's a pity if it hasn't been noted as a new edition, or revision, so that it's clear which copies have been updated and which haven't. There may be a clue on page iv, facing the contents page, which has a row of numbers, decreasing from 10 to 2, at the foot. Perhaps in the first printing there was a "1" in that row, later expunged. I have no idea what these publisher's markings imply, but it might be interesting to discover whether in other copies those numbers are the same. Here's a suggestion; that it would benefit all if, at least, that remaining error on page 206 was communicated to the publisher. Clearly, Herbert's the man to do it. George. ================================================================ contact George Huxtable by email at george---.u-net.com, by phone at 01865 820222 (from outside UK, +44 1865 820222), or by mail at 1 Sandy Lane, Southmoor, Abingdon, Oxon OX13 5HX, UK. ================================================================