Definitions

# Great-circle distance

The great-circle distance is the shortest distance between any two points on the surface of a sphere measured along a path on the surface of the sphere (as opposed to going through the sphere's interior). Because spherical geometry is rather different from ordinary Euclidean geometry, the equations for distance take on a different form. The distance between two points in Euclidean space is the length of a straight line from one point to the other. On the sphere, however, there are no straight lines. In non-Euclidean geometry, straight lines are replaced with Geodesics. Geodesics on the sphere are the great circles (circles on the sphere whose centers are coincident with the center of the sphere).

Between any two points on a sphere which are not directly opposite each other, there is a unique great circle. The two points separate the great circle into two arcs. The length of the shorter arc is the great-circle distance between the points. A great circle endowed with such a distance is the Riemannian circle.

Between two points which are directly opposite each other, called antipodal points, there are infinitely many great circles, but all great circle arcs between antipodal points have the same length, i.e. half the circumference of the circle, or $pi r$, where r is the radius of the sphere.

Because the Earth is approximately spherical (see Earth radius), the equations for great-circle distance are important for finding the shortest distance between points on the surface of the Earth, and so have important applications in navigation.

## The geographical formula

Let $phi_s,lambda_s; phi_f,lambda_f;!$ be the geographical latitude and longitude of two points (a base "standpoint" and the destination "forepoint"), respectively, $Deltalambda;!$ the longitude difference and $Deltawidehat\left\{sigma\right\};!$ the (spherical) angular difference/distance, or central angle, which can be constituted from the spherical law of cosines:

$\left\{color\left\{white\right\}Big|\right\}Deltawidehat\left\{sigma\right\}=arccosbig\left(cosphi_scosphi_fcosDeltalambda+sinphi_ssinphi_fbig\right).;!$

The distance d, i.e. the arc length, for a sphere of radius r and $Deltawidehat\left\{sigma\right\}!$ given in radians, is then:

$d = r Deltawidehat\left\{sigma\right\}.$

This arccosine formula above can have large rounding errors for the common case where the distance is small, however, so it is not normally used. Instead, an equation known historically as the haversine formula was preferred, which is much more accurate for small distances:

$\left\{color\left\{white\right\}frac\left\{bigg|\right\}$ =2arcsinleft(sqrt{sin^2left(frac{phi_f-phi_s}{2}right)+cos{phi_s}cos{phi_f}sin^2left(frac{Deltalambda}{2}right)}right).;!

(Historically, the use of this formula was simplified by the availability of tables for the haversine function: hav(θ) = sin2(θ/2).)

Although this formula is accurate for most distances, it too suffers from rounding errors for the special (and somewhat unusual) case of antipodal points (on opposite ends of the sphere). A more complicated formula that is accurate for all distances is the Vincenty formula:

$\left\{color\left\{white\right\}frac\left\{bigg$
>Deltawidehat{sigma}=arctanleft(frac{sqrt{left(cosphi_fsinDeltalambdaright)^2+left(cosphi_ssinphi_f-sinphi_scosphi_fcosDeltalambdaright)^2}}{sinphi_ssinphi_f+cosphi_scosphi_fcosDeltalambda}right);;!

(When programming a computer, one should use the `atan2()` function rather than the ordinary arctangent function (`atan()`), in order to simplify handling of the case where the denominator is zero.)

If r is the great-circle radius of the sphere, then the great-circle distance is $r,Deltawidehat\left\{sigma\right\};!$.

Note: above, accuracy refers to rounding errors only; all formulas themselves are exact (for a sphere).

## Spherical distance on the Earth

The shape of the Earth more closely resembles a flattened spheroid with extreme values for the radius of arc, or arcradius—the radius of curvature, of 6335.437 km at the equator (vertically) and 6399.592 km at the poles, and having an average great-circle radius of 6372.795 km (3438.461 nautical miles). (Note that "arcradius" or "radius of curvature" is not the distance from the center of the earth to the surface. The distance from the center to the surface is smaller at the poles than at the equator; the arcradius is larger at the poles than at the equator.) Using a sphere with a radius of 6372.795 km thus results in an error of up to about 0.5%.

## A worked example

In order to use this formula for anything practical you will need two sets of coordinates. For example, the latitude and longitude of two airports:

• Nashville International Airport (BNA) in Nashville, TN, USA: N 36°7.2', W 86°40.2'
• Los Angeles International Airport (LAX) in Los Angeles, CA, USA: N 33°56.4', W 118°24.0'

First, convert these coordinates to decimal degrees (Sign × (Deg + (Min + Sec / 60) / 60)) and radians (× π / 180) before you can use them effectively in a formula. After conversion, the coordinates become:

• BNA: $phi_s= 36.12^circapprox 0.6304mbox\left\{ rad\right\};;;lambda_s=-86.67^circapprox -1.5127mbox\left\{ rad\right\};;!$
• LAX: $phi_f= 33.94^circapprox 0.5924mbox\left\{ rad\right\};;;lambda_f=-118.40^circapprox -2.0665mbox\left\{ rad\right\};;!$

Using these values in the angular difference/distance equation:

$r,Deltawidehat\left\{sigma\right\}approx 6372.795times0.45306 approx 2887.259mbox\left\{ km\right\}.;!$

Thus the distance between LAX and BNA is about 2887 km or 1794 miles (× 0.62137) or 1558 nautical miles (× 0.539553).