Isosceles Triangle in a Square

If we extend the altitude of the isosceles $\triangle AEF$ , we find that it is part of the diagonal $AC$ of length $12\sqrt 2$ . If we draw a square with $CE$ and $CF$ as two of the sides, we find that square $A'ECF$ has a diagonal $A'C$ of length $8\sqrt 2$ . Since $\dfrac {A'C}{AC} = \dfrac {8\sqrt 2}{12 \sqrt 2} = \dfrac 23$ , this means that $CE=CF= \dfrac 23 AB = 8$ , $\implies BE=DF=4$ . Then the area of $\triangle AEF$ , $[AEF]=[ABCD]-[ABE]-[ADF] - [CEF] = 144 - 24 - 24 - 32 = \boxed{64}$ .

Let the foot of perpendicular from $A$ to $\overline {FE}$ be $G$ . Then $|\overline {AG}|=8\sqrt 2,|\overline {AC}|=12\sqrt 2,\angle {ACD}=45\degree\implies |\overline {GC}|=|\overline {GF}|=4\sqrt 2$

Hence the area of $\triangle {AEF}$ is $4\sqrt 2\times 8\sqrt 2=\boxed {64}$ square units.

IMHO there is a much simpler way of solving this. WARNING: very basic math ahead

Since $\overline{EF}$ is normal to $\overline{AC}$ we can solve for x (= $\frac{EF}{2}$ ) by via:

$x = \sqrt{2\times12^2}-8\times\sqrt{2}$

and then $x\times 8\times \sqrt{2} = A$

Let $DE = x$ , then using pythagorean theorem twice, we get:

$\left( \dfrac{\sqrt{2(12-x)^2}}{2}\right)^2 + (8\sqrt{2})^2 = \left( \sqrt{x^2 + 12^2}\right)^2$

Solving and simplifying this, and rejecting the negative solution, we have $x = 4$ . Then:

$EF = \sqrt{2(12-4)^2} = 8\sqrt{2}$

Now, the area of triangle an be found easily

$[\triangle AEF] = \dfrac{1}{2} 8\sqrt{2}\cdot 8\sqrt{2} = \boxed{64}$

Diagonal $\overline{BD}$ is parallel to line segment $\overline{FE}$ and is the base of $\triangle DCB$ . Triangle $DCB$ and triangle $CFE$ are similar triangles. Meaning, according to a Similar Triangles Theorem , $\frac{DB}{DF} = \frac{DC}{DE} = \frac{CB}{EF}$ . The height of the isosceles triangle is equal to the sum of half of the square's diagonal and the distance between $\overline{CD}$ and $\overline{FE}$ .

$\small \dfrac{12\sqrt{2}}{2} - 8\sqrt{2} = 2\sqrt{2} \implies \text{The distance between} \space \overline{CD} \space \text{and} \space \overline{FE} = \overline{GE}$

We can now get the leg lengths of $\triangle DFE$ , figure out the length of $\overline{FE}$ , and finally compute for the area of the isosceles triangle.

$\small \begin{aligned} \overline{FD} = & \space \overline{BD} - \overline{BF} \\ \overline{FD} = & \space 12 - \dfrac{\overline{GE}}{\sin45^{\circ}} \\ \overline{FD} = & \space 12 - \dfrac{2\sqrt{2}}{\sin45^{\circ}} \\ \overline{FD} = & \space 12 - 4 \\ \overline{FD} = & \space 8 \end{aligned}$

Next, with the equation $\frac{12}{8} = \frac{12\sqrt{2}}{FE}$ , line segment $\overline{FE}$ is $8\sqrt{2}$ . We can now compute for the needed area: $\dfrac{(8\sqrt{2})^{2}}{2} = \boxed{64}$ .