Staying in the triangle

Let us name the square as ABCD , of length $1$ , and the points to be chosen be $P_{1}, P_{2}, P_{3}$ and $P_{4}$ .

Note : For GIVEN $P_{1}, P_{2}$ , and $P_{3}$ ,Probability = $\frac{\triangle P_{1}P_{2}P_{3})}{Ar(ABCD)}$

Here $4$ cases are possible :

Case - 1 : $P_{1}, P_{2}$ and $P_{3}$ lie on the same edge. The probability that this happens = $\frac{4}{4^3} = \frac{1}{16}$ . Now , after this the probability that $P_{4}$ lies in triangle formed by $P_{1}, P_{2}$ and $P_{3}$ = 0.

Case - 2 : $2$ of $P_{1},P_{1},P_{1}$ (say $P_{1}$ and $P_{2}$ lie on the same edge(say AB) and $P_{4}$ lies on an edge(say BC) adjacent to that edge.

The probability that this happens = $\frac{{3 \choose 2} \times 4 \times 2}{4^3} = \frac{3}{8}$ . Now Let us say $P_{1} , P_{2}$ and $P_{3}$ are at a distance $x , y$ and $z$ from $B$ .

The probability that now $P_{4}$ happens to be inside the triangle formed by them = $\int_{0}^{1}\int_{0}^{1}\int_{0}^{1} \frac{1}{2}|x - y|z dxdydz = \frac{1}{12}$

Case-3 : Two points(Say $P_{1}$ and $P_{2}$ ) lie on the same edge $AB$ , and the third one lies on the opposite vertex.

The probability that this happens = $\frac{{3 \choose 2} \times 4 \times 1}{r^3} = \frac{3}{16}$

Say, distance of $P_{1}$ and $P_{2}$ from $A$ is $x$ and $y$ respectively,

The probability that $P_{4}$ lies in triangle formed by $P_{1} ,P_{2}$ , and $P_{3}$ = $\int_{0}^{1} \int_{0}^{1} \frac{1}{2}|x - y|dxdy = \frac{1}{6}$ (Note that i have divided by $1$ as the area of sample space = $1$ )

Case - 4 : All of $P_{1}, P_{2}$ and $P_{3}$ lie on different edges.

The probability that this happens = $\frac{4 \times 3 \times 2}{4^3} = \frac{3}{8}$

Now, say $P_{1}$ is on $AB$ , $P_{2}$ is on $BC$ , $P_{3}$ is on $AD$ .

Say $BP_{1} = x$ , $BP_{2} = y$ , $AP_{3} = z$ .

Area of triangle formed = $\frac{1}{2}(x + z - (1 - y)z - xy)$

Hence , P( $P_{4}$ lies inside the triangle) = $\int_{0}^{1} \int_{0}^{1} \int_{0}^{1} \frac{1}{2}(x + z - (1 - y)z - xy) dxdydz = \frac{1}{4}$

Summing according to Baye's theorem ,

Required probability = $\frac{3}{8} \times \frac{1}{12} + \frac{3}{16} \times \frac{1}{6} + \frac{3}{8} \times \frac{1}{4} = \frac{5}{32}$ $\Rightarrow a + b = \fbox{37}$

Casework on which sides the points are on. If they are all on the same side (4 cases), the area is trivially 0. If two are on the same side and the other is adjacent to that side (24 cases), the average area is 1/12.

We will quickly show that given two random variables $x,y \in [0,1]$ , the expected value of $|x-y|$ is 1/3. Visualize the search space as a unit square. Then |x-y| can be modeled as two tetrahedra with total area 1/3.

Therefore, the average distance (and hence base) is 1/3. The average height is 1/2 (the third point). So $1/3\cdot 1/2\cdot 1/2 = 1/12$ .

If two are on the same side and the other is opposite to that side (12 cases) the average area is 1/6; the height is guaranteed to be 1 this time.

Otherwise, they are all on different sides (24 cases). By keeping two of the points fixed, it can easily be shown that the average area of the triangle occurs when the third is at the center of the segment. Apply this three times to get an area of 1/4.

So $(1/4\cdot 24+1/6 \cdot 12+1/12\cdot 24+0\cdot 4)/64 = 5/32$ and the answer is 37.

Let $A$ be the event that all three points are chosen on the same edge of the square. Let $B$ be the event that two points are chosen on the same edge of the square, with the third point on the opposite edge. Let $C$ be the event that two points are chosen on the same edge of the square, with the third point chosen on an adjacent edge. Let $D$ be the event that the three points are all on different edges of the triangle. Since each point is equally likely to be chosen on any one of the four edges, we deduce that $\mathbb{P}[A] = \tfrac{1}{16}$ , $\mathbb{P}[B] = \tfrac{3}{16}$ , $\mathbb{P}[C] = \tfrac{6}{16}$ and $\mathbb{P}[D] = \tfrac{6}{16}$ .

Let $I$ be the event that the fourth point is inside the triangle formed by the other three points. If we assume that each side of the square has length $1$ then, for any particular choice of the first three points, the probability of $I$ is the area of the triangle formed by the other three points.

1. In the case of event $A$ , the area formed by the three points is $0$ . Thus $\mathbb{P}[I|A] = 0$ .

2. In the case of event $B$ , let the distances of the two points on the same side from one end of that side be $X$ and $Y$ . Then $X$ and $Y$ are both uniformly distributed on $[0,1]$ , and the area of the triangle is $\tfrac12|X-Y|$ . Thus $\mathbb{P}[I|B] \; = \; \int_0^1 \int_0^1 \tfrac12|x-y|\,dx\,dx \; = \; \frac16$

3. In the case of event $C$ , let the distances of the two points on the same side from one end of that side be $X$ and $Y$ , and let $Z$ be the height of the third point above that side. Then $X$ , $Y$ and $Z$ are each uniformly distributed on $[0,1]$ , and the area of the triangle is $\tfrac12|X-Y|Z$ . Thus $\mathbb{P}[I|C] \; = \; \int_0^1\int_0^1\int_0^1 \tfrac12|x-y|z\,dx\,dy\,dz \; =\; \frac{1}{12}$

4. In the case of event $D$ , let the square be $PQRS$ , and suppose that the three vertices $V_1,V_2,V_3$ lie on $PQ$ , $QR$ and $RS$ respectively. This is always possible if we label the vertices correctly. Let $X$ , $Y$ and $Z$ be the distances $PV_1$ , $QV_2$ and $RV_3$ respectively. Then $X,Y,Z$ are uniformly distributed on $[0,1]$ and the area of the triangle is $1 - \tfrac12(1-X)Y - \tfrac12(1-Y)Z - \tfrac12(X+1-Z)$ Thus $\begin{array}{rcl}\mathbb{P}[I|D] & = & \int_0^1\int_0^1\int_0^1 \tfrac12\left[\begin{array}{c}1 - x - y \\ + xy + yz \end{array}\right]\,dx\,dy\,dz \; = \; \frac14 \end{array}$

Thus we deduce that $\mathbb{P}[I] \; = \; \tfrac{1}{16}\times0 + \tfrac{3}{16}\times\tfrac16 + \tfrac{6}{16}\times\tfrac{1}{12} + \tfrac{6}{16}\times\tfrac14 \; = \; \tfrac{5}{32}$ and hence the answer is $5 + 32 = 37$ .

Consider a unit square with length 1. Define

$\begin{aligned} \vec{x}_1,\: \vec{x}_2,\: \vec{x}_3:&&&\text{points on the boundary of the square, generate a triangle with area }A\\ \vec{x}:=(x_1,\:x_2,\:x_3):&&&\text{random variables that control the position of }\vec{x}_i\text{ on the border with distribution }P_{\vec{x}}(\vec{x})\\ \vec{p}:&&&\text{fourth point in the interior of the square}\\ E:&&&\text{event that }\vec{p}\text{ lies within the triangle} \end{aligned}$

If the triangle has already been chosen, we can compare the area of the trangle and the unit square: $P(E|\vec{x})=\frac{A(\vec{x})}{1}=A(\vec{x})$

With Bayes' Theorem, we can calculate $P(E)$ . The set $\Omega$ contains all values for the random variables $x_i$ : $P(E)=\int_\Omega P(E|\vec{x})P_{\vec{x}}(\vec{x})\:dV=\int_\Omega A(\vec{x})P_{\vec{x}}(\vec{x})\:dV=\mathbb{E}(A(\vec{x}))$

---

Let's analyse what happens when we place $\vec{x}_i$ randomly on the square's boundary. We can break up the process into two steps:

1. For each of the three $\vec{x}_i$ , choose one of the four square's sides. Each side has equal probability
2. Place $\vec{x}_i$ randomly on its side. Its position is independently and uniformly distributed

Think about the first step: There are a total of $4^3$ choices of equal probability $\frac{1}{4^3}$ ! Luckily, we can order the choices that generate a triangle with area $A>0$ into just three distinct classes:

In the second step after choosing a side, we assign to each point $\vec{x}_i$ a uniformly distributed random variable $x_i\in[0;\:1]$ that controls its position on its side (check the graphics). Now we are ready to tackle the three classes.

---

• $E_1:\quad\vec{x}_i$ lie on different sides (left graphic)

  There are 4 choices for the empty border and afterwards $3!=6$ choices for the order of the three points. By rotation, all 24 choices are the same! With the coordinate system in the graphic, we get $\begin{aligned} A(\vec{x})&= \frac{1}{2}\left| \begin{pmatrix}x_3-x_1\\1\\0\end{pmatrix} \times \begin{pmatrix}-x_1\\x_2\\0\end{pmatrix} \right|=\frac{1}{2}(x_1(1-x_2)+x_2x_3)\\[1.75em] P(E_1) &= \frac{24}{4^3}\mathbb{E}[A(\vec{x})]=\frac{3}{8}\cdot\frac{1}{2}\mathbb{E}[x_1(1-x_2)+x_2x_3]\\[1em] &= \frac{3}{16}\left( \mathbb{E}[x_1](1-\mathbb{E}[x_2])+\mathbb{E}[x_2]\mathbb{E}[x_3]] \right)\underset{\red{(*)}}{=}\frac{3}{16}\left(\frac{1}{2}\cdot\frac{1}{2} + \frac{1}{2}\cdot\frac{1}{2} \right)=\frac{3}{32} \end{aligned}$

• $E_2:\quad$ two points share a side and the third lies on an adjacent side (center graphic)

  There are 4 choices for the border with two points, 2 choices for the border of the third point and afterwards 3 possibilities to pick one $\vec{x}_i$ as the third point. By rotation and mirroring, all 24 choices are are the same! The graphic in the middle yields $\begin{aligned} A(\vec{x}) &= \frac{1}{2}|x_3-x_1|y_2\\[1em] P(E_2) &= \frac{24}{4^3}\mathbb{E}[A(\vec{x})]=\frac{3}{8}\cdot\frac{1}{2}\mathbb{E}[|x_3-x_1|]\mathbb{E}[x_2]\underset{\red{(*)}}{=}\frac{3}{16}\cdot\frac{1}{3}\cdot\frac{1}{2}=\frac{1}{32} \end{aligned}$

• $E_3:\quad$ two points share a side and the third lies on the opposite side (right graphic)

  There are 4 choices for the border with two points and afterwards 3 possibilities to pick one $\vec{x}_i$ as the third point on the opposite side. After rotation, all 12 choices are are the same! Using the right graphic: $\begin{aligned} A(\vec{x}) &= \frac{1}{2}|x_3-x_1|\cdot 1=\frac{1}{2}|x_3-x_1|\\[1em] P(E_3) &= \frac{12}{4^3}\mathbb{E}[A(\vec{x})]=\frac{3}{16}\cdot\frac{1}{2}\mathbb{E}[|x_3-x_1|]\underset{\red{(*)}}{=}\frac{3}{32}\cdot\frac{1}{3}=\frac{1}{32} \end{aligned}$

Combine the three distinct classes: $P(E)=P(E_1)+P(E_2)+P(E_3)=\frac{5}{32}\quad\Rightarrow\:\:\:\: a+b=5+32=\boxed{37}$

---

Rem: In $\red{(*)}$ , we used the fact that for independent and uniformly distributed variables $x_i,\:x_j\in [0;\:1]$ : $\begin{aligned} \mathbb{E}[x_i]&=\int_0^1x_i\cdot 1\:dx_i=\frac{1}{2},&&&\mathbb{E}[|x_i-x_j|]=\int_{(x_i,\:x_j)\in[0;\:1]^2}|x_i-x_j|\cdot 1\:d(x_i;\:x_j)=\frac{1}{3} \end{aligned}$

First, we place the first three points on the sides of the squares. There are 4 cases:

Case 1) All points on the same side: After placing the first point, there is a $\frac{1}{4}\times\frac{1}{4}=\frac{1}{16}$ chance that the next two points will be placed on the same side.

Case 2) Two points on the same side and one on an adjacent side: After placing the first point, there is a $\frac{1}{4}\times\frac{1}{2}=\frac{1}{8}$ chance that the second point is placed on the same side and the last point is placed on an adjacent side, however, the lone point could have been placed either first, second, or third, so this case has a total probability of $3\times\frac{1}{8}=\frac{3}{8}$ .

Case 3) Two points on the same side and one point on the opposite side: After placing the first point, there is a $\frac{1}{4}\times\frac{1}{4}=\frac{1}{16}$ chance that the second point is placed on the same side and the last point is placed on the opposite side, however, the lone point could have been placed either first, second, or third, so this case has a total probability of $3\times\frac{1}{16}=\frac{3}{16}$ .

Case 4) All points being placed on different sides: After placing the first point, the second point can be placed on any of the remaining 3 sides, and the last point can be placed on any of the remaining 2 sides, so this case has a total probability of $\frac{3}{4}\times\frac{2}{4}=\frac{3}{8}$ .

We now consider the fourth point. The probability that this point lands in the triangle determined by the first three points is the area of success divided by total area, or the area of the triangle divided by the area of the square. We can assume that the square is the unit square to simplify calculations. We now consider each case separately.

Case 1) All points on the same side: Clearly if all points are on the same side the resulting triangle has no area, so the probability of success by case 1 is $\frac{1}{16}\times0=0$ .

Case 2) Two points on the same side and one on an adjacent side: The base of the triangle is determined by the distance between the two points on the same side. The height is determined by the perpendicular distance of the third point from the side on which the other two points lie. Assume WLOG that the first point is closer to the corner formed by the two edges on which the points lie than the second. If x is the distance between this corner and the first point, y is the distance between the second point and the first point, and z is the perpendicular distance between the third point and the edge on which the other two points lie, we evaluate the integral $\int_0^1 \int_0^1 \int_x^1 \frac{1}{2}yz dy dx dz = \frac{1}{12}$

Case 3) Two points on the same side and one point on the opposite side: We write the expression for the base similarly, however, in this case the height is fixed at 1. Thus, the integral is $\int_0^1 \int_x^1 \frac{1}{2}y dy dx = \frac{1}{6}$

Case 4) All points being placed on different sides: Orient the square such that one point lies on the positive x axis, one on the positive y axis, and the last on the side opposite the y axis. Let x be the distance between the origin and the first point, let y be the distance between the origin and the second point, and let z be the distance between the final point and the x axis (Drawing the square is especially useful in this case). The area of the triangle of interest is $1-\frac{1}{2}xy-\frac{1}{2}(1-x)z-\frac{1}{2}(y-z)-(1-y)$ which simplifies to $\frac{1}{2}(xz-xy+y)$ The integral becomes, $\int_0^1 \int_0^1 \int_0^1 \frac{1}{2}(xz-xy+y) dx dy dz = \frac{1}{4}$

Thus, the probability that the fourth point lies within the interior of the triangle determined by the other 3 is $0+\frac{3}{8}\times\frac{1}{12}+\frac{3}{16}\times\frac{1}{6}+\frac{3}{8}\times\frac{1}{4} = \frac{5}{32}$

So the answer is $5+32=\boxed{37}$

WLoG: The square has sides of length 1.

The probability that the interior point is in the triangle formed by the other 3 points is now equal to the area of the triangle.

There are 4 different ways for 3 points to be chosen on the 4 sides of the square (with a total of 64 possibilities). The probability that one or more of the points are on a corner of the square is negligible and those cases can be safely ignored:

1) All 3 on the same side (4 possibilities)

The area of the triangle in this case is always 0.

2) 2 on the same side and the other point on an adjacent side (24 possibilities);

Let points X and Y be on the same side and point Z be on an adjacent side, with X at distance $x$ , Y at distance $y$ , and Z at distance $z$ from the corner shared by both sides. The area of the triangle in this case is $\frac{1}{2} z |x-y|$ . Because of the uniformly chosen points all triangles of this form have the same probability, so the average area of these triangles is $\int_0^1 \int_0^1 \int_0^1 \! \frac{1}{2}z|x-y| \, \mathrm{d}z \, \mathrm{d}y \, \mathrm{d}x = \frac{1}{12}$ .

3) 2 on the same side and the other point on the opposite side (12 possibilities);

Let points X and Y be on the same side and point Z be on the opposite side, with X at distance $x$ , Y at distance $y$ , and Z at distance $z$ from the same unoccupied side of the square. The area of the triangle in this case is $\frac{1}{2}|x-y|$ , and the average area of all these triangles is $\int_0^1 \int_0^1 \int_0^1 \! \frac{1}{2}|x-y| \, \mathrm{d}z \, \mathrm{d}y \, \mathrm{d}x = \frac{1}{6}$ .

4) All 3 on different sides (24 possibilities).

Let point X and Y be on opposite sides of the square and point Z be on one of the other two sides, with X at distance $x$ , and Y at distance $y$ from the side on which Z is on, and Z at distance $z$ from the side on which X is on. The area of the triangle in this case is the area of the right-angled trapezium formed by the Z-side, the Z-side parts of the X and Y sides and XY ( $\frac{1}{2}(x+y)$ ) minus the area of the two right-angled triangles between X and Z ( $\frac{1}{2}xz$ ) and Y and Z ( $\frac{1}{2}y(1-z)$ ). The area is $\frac{1}{2}x-\frac{1}{2}xz+\frac{1}{2}yz$ , and the average area of all these triagles is $\int_0^1 \int_0^1 \int_0^1 \! \frac{1}{2}x-\frac{1}{2}xz+\frac{1}{2}yz \, \mathrm{d}z \, \mathrm{d}y \, \mathrm{d}x = \frac{1}{4}$ .

The probability that the interior point is in the triangle formed by the other 3 points is now: $\frac{4\times0+24\times\frac{1}{12}+12\times\frac{1}{6}+24\times\frac{1}{4}}{64}=\frac{5}{32}.$

The answer is therefore $5+32=\fbox{37}$ .