A Triangle Problem

Label the equilateral triangle $QRS$ and label the point outside the triangle $P$ , and rotate the whole diagram $60°$ counterclockwise about $Q$ so that $S$ goes to $R$ , $P$ goes to $P'$ , and $R$ goes to $R'$ . Then $RP' = SP = 3$ , $R'P' = RP = 5$ , and $QP = QP' = 7$ .

Draw $PP'$ . Since $QP = QP'$ and $\angle PQP' = 60°$ , $\triangle PQP'$ is an equilateral triangle and $PP' = QP = QP' = 7$ .

By the law of cosines on $\triangle PRP'$ , $\angle PRP' = \cos^{-1} (\frac{3^2 + 5^2 - 7^2}{2 \cdot 3 \cdot 5}) = 120°$ .

By the angle sum of a triangle, $\angle RPP' + \angle RP'P = 60°$ , and from equilateral triangle $\triangle PQP'$ , $\angle QP'R + \angle RP'P = \angle QPR + \angle RPP' = 60°$ , and since $\triangle QP'R \cong \triangle QPS$ by SSS congruence, $\angle QPS = \angle QP'R$ . Therefore, $\angle SPR = \angle QPS + \angle QPR = 60°$ .

Finally, by law of cosines on $\triangle SPR$ , $s^2 = 3^2 + 5^2 - 2 \cdot 3 \cdot 5 \cdot \cos 60° = \boxed{19}$ .

Let the coordinate of the three vertices of the equilateral triangle be $\left(0, \frac {\sqrt 3}2s \right)$ , $\left(\frac 12s, 0 \right)$ , and $\left(-\frac 12s, 0 \right)$ , and the coordinates of the external point be $P (x,y)$ . Then by Pythagorean theorem :

$\begin{cases} x^2 + \left(y - \frac {\sqrt 3}2s\right)^2 = 3^2 & ...(1) \\ \left(x - \frac 12s\right)^2 + y^2 = 5^2 & ...(2) \\ \left(x + \frac 12s\right)^2 + y^2 = 7^2 & ...(3) \end{cases} \implies \begin{cases} x^2 + y^2 - \sqrt 3sy + \frac 34 s^2 = 9 & ...(1) \\ x^2 - sx + \frac 14 s^2 + y^2 = 25 & ...(2) \\ x^2 + sx + \frac 14 s^2 + y^2 = 49 & ...(3) \end{cases}$

$\begin{cases} (3)-(2): \ 2sx = 24 & \implies x = \dfrac {12}s \\ (3)-(1): sx + \sqrt 3sy - \dfrac {s^2}2 = 40 & \implies y = \dfrac {28}{\sqrt 3s} + \dfrac s{2\sqrt 3} \end{cases}$

Substitute $x$ and $y$ in $(3)$ :

$\begin{aligned} \frac {144}{s^2} + 12 + \frac {s^2}4 + \frac {784}{3s^2} + \frac {28}3 + \frac {s^2}{12} & = 49 \\ \frac {s^2}3 - \frac {83}3 + \frac{1216}{3s^2} & = 0 & \small \color{#3D99F6} \text{Multiply both sides by }3s^2 \\ s^4 - 83s^2 + 1216 & = 0 \\ (s^2 - 64)(s^2 - 19) & = 0 \\ \implies s^2 & = \boxed{19} \end{aligned}$

Note that $s=\sqrt {64} = 8$ is unacceptable because the point $P$ will be in the equilateral triangle.

Let ABC be the given equilateral triangle and P the point who’s distances from the vertices of ABC are PA=7, PB=3 and PC=5. If we consider the equilateral triangles PBD and PCE outside BPC, we get the well-known figure where Fermat’s point occurs as the point where the red line segments concur. Now, these red segments are congruent, just because triangles ABP and CBD are congruent as well as triangles DPC and BPE are. Hence, the sides of triangle BPE have lengths 3, 5 and 7. Using cosine rule on this triangle, we get: $\begin{array}{l} B{E^2} = P{B^2} + P{E^2} - 2 \cdot PB \cdot PE \cdot \cos \theta \\ \Rightarrow \cos \theta = \frac{{P{B^2} + P{E^2} - B{E^2}}}{{2 \cdot PB \cdot PE}} = \frac{{{3^2} + {5^2} - {7^2}}}{{2 \cdot 3 \cdot 5}} = - \frac{1}{2}\\ \Rightarrow \theta = 120^\circ \\ \Rightarrow \angle BPC = \theta - \angle EPC = 120^\circ - 60^\circ = 60^\circ \end{array}$ Once again, by cosine rule, on triangle BPC this time, we have: $\begin{array}{l} B{C^2} = P{B^2} + P{C^2} - 2 \cdot PB \cdot PC \cdot \cos \left( {\angle BPC} \right)\\ = {3^2} + {5^2} - 2 \cdot 3 \cdot 5 \cdot \cos 60^\circ = 9 + 25 - 2 \cdot 3 \cdot 5 \cdot \frac{1}{2} = 19 \end{array}$ The answer is ${s^2} = \boxed{19}$

Note: triangle A' is obtained upon a 60 degrees clockwise rotation of triangle A; Theta is the angle ∠OPN

Inspiration: Coffin Problems

I use GeoGebra . The Answer is 19.

Use the relation 3(3^4+5^4+7^4+s^4)=(3^2+5^2+7^2+s^2)^2 Find, s^2=19 and 64. Choose the relevant Answer =19.