Random points on circle

The first point can just be placed wherever you want, because it is a circle. The second point can be placed between 0 degrees and 180 degrees, because more than 180 degrees is the same but mirrored. The average of 0 and 180 degrees is 90 degrees. So now you have the two points 90 degrees from each other, on average. For the center point to be in the triangle, the last point has to be between the other side of the first two points.

Because the last point has to be in that quarter, the answer is 1/4.

We can assume that the first point lies at the bottom of the circle and that the second lies on the right of the circle, as other solutions have explained. The center is in the triangle if the third point lies in the antipodal arc generated by the first two points (see other solutions for visualization).

Note that if the first two points are displaced by an angle $\theta$ , the probability that the center is in the triangle is given by $p(\theta) = \frac{\theta}{2\pi^2}$ . Note that $\theta \in [0,\pi]$ . Now, we integrate to find across all $\theta$ . Let $a$ be the probability that the center is in the triangle:

$a = \int_0^{\pi}p(\theta)\hspace{1mm}d\theta = \int_0^{\pi}\frac{\theta}{2\pi}\hspace{1mm}d\theta = \frac{\pi^2}{4\pi^2} = \frac{1}{4}$

For a great animated solution you can check out this 3Blue1Brown video.