Round and round we go .....

We want none of the two points to lie in the same hemisphere.

It doesn't matter where the first point goes, so suppose that the first point is at the North pole of the sphere. Let $A$ be the angle subtended at the centre of the sphere by the first two points. If $A \le u$ where $0 \le u \le \pi$ , then the second point must lie in a "polar cap" of area $\int_0^u \int_0^{2\pi} \sin\theta\,d\phi\,d\theta \; =\; 2\pi(1 - \cos u)$ and hence $P[A \le u] = \tfrac12(1 - \cos u)$ , and hence $A$ has probability density function $f_A(u) \; = \; \left\{ \begin{array}{lll} \frac12\sin u & \hspace{1cm} & 0 \le u \le \pi \\ 0 & & \mathrm{o.w.} \end{array} \right.$ Suppose now that $A = u > \tfrac12\pi$ , so that the first two points are in different hemispheres. Rotate the point of view so that both points lie on the equator. Then the third point will not be in the same hemisphere as either of the two points provided that it lies in an "orange-slice" shaped segment that has longitudinal width $\pi-u$ . Thus the probability that all three points lie in different hemispheres is $\int_{\frac12\pi}^\pi f_A(u) \tfrac{\pi -u}{2\pi}\,du \; =\; \tfrac{1}{4\pi}\int_{\frac12\pi}^\pi (\pi-u)\sin u\,du \; = \; \tfrac{1}{4\pi}$ which makes the answer $1 + 4 = \boxed{5}$ .

I put the unit sphere on a Cartesian coordinate system centered at the origin. I then assumed that the first of the three points was on the south pole (i.e., the point $(0,0,-1)$ ). Note: I do not lose any generality by doing this.

Then the next (second) point can only be located at any given point on the top half of the sphere. This means that the probability that the second point is not within an arc length of $\frac{\pi}{2}$ of the first point is $\frac{1}{2}$ . Next I find the probability the third point is not within an arc length of $\frac{\pi}{2}$ of the other two points and multiply this by $\frac{1}{2}$ to get the probability that none of the points are within an arc length of $\frac{\pi}{2}$ of any other point.

So, I will turn my attention to the subset of the top half of the sphere where the third point would not be within an arc length of $\frac{\pi}{2}$ of the second point. Let the coordinates of the second point be given by $(x,y,z)$ . Consider the plane through the origin perpendicular to the vector $<x,y,z>$ . It intersects the unit sphere in a great circle that divides the sphere into two halves where one of the halves is a restricted region for the third point.

The great circle that lies in the xy-plane and the aforementioned great circle form a region that the third point must be confined to. This region is a diangle. Its area is $2\theta$ , where $\theta$ is the angle between the planes.

The angle of interest between the planes is the same as the angle between the vectors $(-x,-y,-z)$ (negation of the location of the second point) and $(0,0,-1)$ (south pole). I found the angle between the vectors using the two formulas for the dot product: $<-x,-y,-z>\cdot<0,0,-1>=\sqrt{x^2+y^2+z^2}\sqrt{0^2+0^2+(-1)^2}\cos\theta \Rightarrow \theta=\cos^{-1}z$ .

Therefore, the area of the region the third point is restricted to is $2\cos^{-1}z.$ For each planar slice (where the plane is parallel to the xy-plane), the distance around the unit sphere is $2\pi\sqrt{1-z^2}$ . If we divide $2\pi\sqrt{1-z^2}$ by the surface area of the top half of the sphere, $2\pi,$ and multiply it by $ds,$ it tells us the proportion of the area of the infinitesimal strip to the area of the whole top half of the sphere. I then multiply this by the proportion of the area of the diangle, $2\cos^{-1}z$ , to the surface area of the whole sphere, $4\pi$ , and integrate the result to find the probability that the third point is not within an arc length of $\frac{\pi}{2}$ of any other point.

But, first, I must find $ds$ . I have designed my own approach for solving this problem (this particular method may already exist elsewhere though or it may even have a flaw). The following equation describes each circular slice parallel to the xy-plane of the top half of a sphere: $(x,y,z)=(\sqrt{1-z^2}\cos\phi,\sqrt{1-z^2}\sin\phi,z)$ corresponding to each value of $z$ from $0$ to $1$ . I will perturb $z$ by $\Delta z$ and note that the set of mappings $M_{\Delta z}$ that map $(\sqrt{1-z^2}\cos\phi,\sqrt{1-z^2}\sin\phi,z)$ to $(\sqrt{1-(z+\Delta z)^2}\cos\phi,\sqrt{1-(z+\Delta z)^2}\sin\phi,z+\Delta z)$ have the property that $\lim_{\Delta z\to 0} M_{\Delta z}=M_0=I$ , where $I$ is the identity mapping. Then I claim that $\frac{ds}{dz}=\lim_{\Delta z\to 0} \Big|\frac{M_{\Delta z}-M_0}{\Delta z}\Big|$ . Using this possibly questionable definition (and bringing the limit inside the magnitude bars), I find $\frac{ds}{dz} = |<\frac{-z}{\sqrt{1-z^2}}\cos\phi,\frac{-z}{\sqrt{1-z^2}}\sin\phi,1>|=\frac{1}{\sqrt{1-z^2}}$ .

Therefore, $\frac{1}{2\pi}\int_0^1 \cos^{-1}z\sqrt{1-z^2}ds=\frac{1}{2\pi}\int_0^1 \cos^{-1}z\sqrt{1-z^2}\frac{ds}{dz}dz=\frac{1}{2\pi}\int_0^1 \cos^{-1}zdz=\frac{1}{2\pi}[x\cos^{-1}x-(1-x^2)^\frac{1}{2}]_0^1=\frac{1}{2\pi}$ . Multiplying by the $\frac{1}{2}$ I mentioned at the beginning gives the final answer of $\frac{1}{4\pi}$ .