Spherical trigonometry and navigation

Setting the scene

A ship leaves New York, USA, bound for Brest, France. Assuming it takes the shortest route, what will be its departure course? If it is sailing at 15 knots, how long will it take to reach Brest? The coordinates of the two ports are known (below).

This simple question has been at the heart of navigation problems for millennia. Given two known points on earth, what is the distance between them, and what course should we take to get from one point to the other? In these modern times, it’s very tempting to get the answer from Google. But then again, how does it get that answer?

Believe it or not, the method of calculating the course to take between two known points was first solved to meet religious needs. Muslims wanted to know how to orient themselves so as to pray facing Mecca. Mathematicians came up with an answer… and today it forms part of the equations used in astronavigation.

What we know

Four essential pieces of information are known about this problem:

• The radius of the earth, which is 6,378 kilometers.
• The distance between New York and the North Pole: Latitude is measured from the equator, so the angle between New York and the North Pole is 90° – [latitude]:90° - 40.730610° \approx 49.269°. This angle can be converted into a distance by measuring the associated fraction of the circumference, which gives 5495 kilometers (2\pi\cdot6378\frac{49.269}{360}\approx 5495).
• Similarly, the distance between Brest and the North Pole is 41.610° (or 4632 km).
• The angular difference, measured from the North Pole, between New York and Brest. This corresponds to the difference in longitude between the two cities, i.eLHA\approx 69.445°…

This information is used to construct a spherical triangle, the details of which are shown in the image below. Side “c”, between New York and the North Pole, corresponds to the 49.27° degrees calculated above (rounded off). Side “b” corresponds to the distance between Brest and the North Pole (41.61° rounded off). Finally, the longitude difference is 69.45° (rounded) and corresponds to the angle between segments “c” and “b”.

Two pieces of information are sought from this triangle: side “a”, which is the unknown distance between New York and Brest, and the angle between sides “c” and “a”, which corresponds to the gap between true north and the route to Brest. This angle has, of course, a better-known name: the course to take.

Applying the equations

Identifying distance

The most important spherical trigonometry equation is the one linking two sides and an angle to the opposite side:

\cos(a) = \cos(b)\cos(c) \sin(b)\sin(c)\cos(LHA),

where a,b,c are the sides of the spherical triangle (in degrees) and LHA is the difference in longitude between the two cities. The functions \cos(\cdot) and \sin(\cdot) are the usual trigonometric functions. So, by simple substitution, we can calculate the value of \cos(a):

\begin{align*}
\cos(a)&= \cos(49.269)\cos(41.610) \sin(49.269)\sin(41.610)\cos(69.445)\\
&\approx 0.6645.
\end{align*}

Taking the inverse function (\cos^{-1}(0.6645)), we can then deduce that a \approx 48.356°, or 5383 kilometers, or 2906.5 nautical miles.

Dividing this distance (in nautical miles) by the ship’s speed (15 knots) gives 193.77 hours, or 8 days, 1 hour and change.

Identifying the heading

Once the missing side “a” has been identified, we can apply the same equation to identify the heading. However, it’s more convenient to modify the letters to match those on the image. Noting the heading \theta_Nwe have :

\cos(b) = \cos(a)\cos(c) \sin(a)\sin(c)\cos(\theta_N),

A little algebraic reorganization to isolate the unknown yields:

\begin{align*}
\cos(\theta_N)&= \frac{\cos(b)-\cos(a)\cos(c)}{\sin(a)\sin(c)},\\
&\approx \frac{\cos(41.610) - \cos(48.356)\cos(49.269)}{\sin(48.356)\sin(49.269)},\\
&\approx 0.5547.
\end{align*}

Consequently, the course to take is given by the inverse of the cosine of this value, i.e \cos^{-1}(0.5547)\approx 56.310°.

By starting our navigation at heading 056°, we’ll be heading in the direction of Brest. We must remember, however, that this is only the starting heading, as the shortest route is not a constant heading. We’ll have to adjust our course every day (or at any other interval).

Conclusion

Leaving New York, we’d have to sail on a heading of 056° (and make corrections afterwards). After eight days and 1 hour, a ship travelling at 15 knots would cross the Atlantic and arrive in Brest, France.

The clever ones will have noticed that the shortest route between New York and Brest passes over Long Island (New York State) and the State of Massachusetts. So we’d have to make a few modifications, perhaps with a coordinate far enough away from New York harbor, before we could undertake a minimum-distance route. Just because a route is the result of an equation doesn’t mean it doesn’t deserve to be double-checked!

In addition to this practical adjustment, the main purpose of the example is to illustrate an application of spherical trigonometry to sea route calculations. For two given coordinates, you can always calculate the distance and course to take. So practical is this that online sites now incorporate these calculations to identify distances between any number of cities.

What does this have to do with astronomical navigation? Instead of taking the coordinate of a city, we take the coordinate of the foot of a star, i.e. the point on earth that intersects with the line running from the center of the earth to the star. What’s more, in astronomical navigation, we know the distance and direction of the star from the boat (measured with a sextant). The reverse calculation is then required: given a known course and distance to the star, what is the boat’s position? All that’s left to do is get a sextant!

Did you like this text? You can read more in the Learn section.