3-5-7 square point

Let the side length of the square $ABCD$ be $a$ and $A$ be $(0,a)$ , $B(a,a)$ , $C(a,0)$ , $D(0,0)$ and $P(x,y)$ .

Then, we have:

$\begin{cases} PD^2: x^2+y^2 = 25 &...(1)\\ AP^2: x^2+(a-y)^2 = 9 &...(2)\\ PB^2: (a-x)^2+(a-y)^2 = 49 &...(3) \end{cases}$

$\begin{cases} Eq.1-Eq.2: -a^2+2ay = 16 & y = \dfrac {a^2+16}{2a}\\ Eq.3-Eq.2: a^2-2ax = 40 & x = \dfrac {a^2-40}{2a} \end{cases}$

Substituting $x$ and $y$ in $Eq.1$ :

$\Rightarrow \left( \dfrac {a^2-40}{2a} \right)^2 + \left( \dfrac {a^2+16}{2a} \right)^2 = 25\quad \Rightarrow a^4-74a^2+928 = 0$

$\Rightarrow a^2 = \dfrac {74\pm \sqrt{74^2-4(928)}}{2} = \dfrac {74\pm 42}{2} = 58 \space$ or $\space 16 \space$ but $\space a > 4$ .

Therefore, the area of $ABCD = a^2 = \boxed{58}$ .

Let $\triangle AQB \cong \triangle APD$ the $AQ=AP=3$ and $\angle PAQ=90°$ .

As $\triangle APQ$ is isosceles $\angle APQ=45°$

Then by law of cosines proof that $\angle QPB=45°$ so $\angle APB=90°$ . \ $\triangle APB$ is rectangle and $AB$ is its hypotenuse, and $ABCD$ area is $AB^2$ $\therefore AP^2+PB^2=3^2+7^2=AB^2=\boxed{58}$

I will firstly label a few points of interest on the picture:

Let us call the angle $\angle OAB = \theta$ , and thus since $\angle CAB = 90^ \circ$ , we find that angle $\angle OAC = 90^ \circ - \theta$ . Also, let $l$ be the size of the side of the square.

Firstly, applying the law of cosines on the triangle $\triangle{OAB}$ with respect to the side $\overline{OB}$ will yield the following:

$5^{2} = 3^{2} + l^{2} - 2*3*l*cos(\theta)$

$6l*cos(\theta) = l^{2} - 16$

$cos(\theta) = \frac{l^{2} - 16}{6l}$

Now, applying the law of cosines on the triangle $\triangle{OAC}$ with respect to the side $\overline{OC}$ will yield the following:

$7^{2} = 3^{2} + l^{2} - 2*3*l*cos(90^ \circ - \theta)$

$6l*cos(90^ \circ - \theta) = l^{2} - 40$

$cos(90^ \circ - \theta) = \frac{l^{2} - 40}{6l}$

Thus, we can write:

$sin(\theta) = \frac{l^{2} - 40}{6l}$

Now, applying the trigonometric identity $sin^{2}(\theta) + cos^{2}(\theta) = 1$ , we will obtain:

$\frac{l^{4} - 32l^{2} + 256}{36l^{2}} + \frac{l^{4} - 80l^{2} + 1600}{36l^{2}} = 1$

$2l^{4} - 148l^{2} + 1856 = 0$

$l^{4} - 74l^{2} + 928 = 0$

Completing the square:

$l^{4} - 74l^{2} + 1369 = 441$

$(l^{2} - 37)^{2} = 21^{2}$

So, either $l^{2} - 37 = 21$ , which gives us $l^{2} = 58$ , or $l^{2} - 37 = 21$ , which gives us $l^{2} = 16$ . $l^{2}$ cannot be equal to $16$ because if it were, then $l = 4$ , and this would imply that the triangle $\triangle{OAC}$ is degenerate since the triangle inequality wouldn't hold, given that $3 + 4 > 7$ is false. Thus, $l^{2} = 58$ , which so happens to be the area of the square.

Applying Law of Cosines three times on triangles $\triangle APB$ , $\triangle APD$ and $BPD$ we get if we let $AP=a$ , $BP=b$ , $DP=c$ and $AB=L$ :

$L^2=\dfrac{b^2+c^2+\sqrt{(2bc)^2-(b^2+c^2-2a^2)^2}}{2}$

Substitute the known values:

$L^2=\dfrac{b^2+c^2+\sqrt{(2bc)^2-(b^2+c^2-2a^2)^2}}{2} \\ L^2=\dfrac{7^2+5^2+\sqrt{(2\cdot 7\cdot 5)^2-(7^2+5^2-2\cdot 3^2)^2}}{2} \\ L^2=\dfrac{74+\sqrt{1764}}{2} \\ L^2=\boxed{58}$