Minimum area triangle encompassing two circles

Considering the trapezium $PQHF$ , we know that $FH = DG = 2\sqrt{35}$ . By considering similar right-angled triangles, we have $GB = HB = 5\sqrt{35}$ .

If $AE = AF = u$ and $CD = CE = v$ , then the triangle $ABC$ has area $\Delta$ and semiperimeter $s$ , where $\Delta \; = \; \sqrt{7\sqrt{35} \times suv} \hspace{2cm} s \; =\; u + v + 7\sqrt{35}$ Since the inradius of $ABC$ is $7$ we deduce that $\Delta = 7s$ , and hence $\sqrt{7}s = \sqrt{5}uv$ . It is clear that $s$ , and hence $\Delta = 7s$ , will be minimized when $u=v=w$ where $\begin{aligned} \sqrt{5}w^2 & = \; \sqrt{7}(2w + 7\sqrt{35}) = 2w\sqrt{7} + 49\sqrt{5} \\ w^2 - 2\sqrt{\tfrac75}w + 49 & = \; 0 \\ \left(w - \sqrt{\tfrac75}\right)^2 & = \; \tfrac75 \times36 \end{aligned}$ and hence $w = 7\sqrt{\tfrac75}$ . Thus we deduce that the least possible area of $ABC$ is $343 \sqrt{\tfrac75} = \boxed{405.8430731...}$ .

The centers of the two circles lie on the bisector of $\angle A$ . If $t$ is half of $\angle A$ , then it is easy to show that

$\sin t = \dfrac{r_2 - r_1 }{r_2 + r_1} = \dfrac{1}{6}$

where $r_1 = 5 , r_2 = 7$ .

The length of the red line is $L = r_1 + \dfrac{r_1}{\sin t } + r_2 = 42$

Next connect the center of the bigger circle to its tangency point with $BC$ , and let the angle between this segment (green) and the extension of the red line be $\theta$ . It follows that the area of the triangle is given by

$\Delta = L^2 \cos t \sin t + {r_2}^2 ( \tan( \dfrac{\pi}{4} + \frac{1}{2} (t - \theta) ) + \tan( \dfrac{\pi}{4} + \frac{1}{2} (t + \theta) ))$

Differentiating the area with respect to $\theta$ :

$\dfrac{d \Delta}{d \theta} = {r_2}^2 ( - \frac{1}{2} \sec^2 ( \dfrac{\pi}{4} + \frac{1}{2}( t - \theta) ) + \frac{1}{2} \sec^2 ( \dfrac{\pi}{4} + \frac{1}{2} (t +\theta)))$

Equating this to zero, gives us the optimal value of $\theta = 0$ , which clearly minimizes $\Delta$ . With $\theta = 0$ , the expression for $\Delta$ becomes

$\Delta = L^2 \cos t \sin t + 2 {r_2}^2 \tan( \dfrac{\pi}{4} + \dfrac{t}{2} )$

Evaluating this, results in $\Delta \approx 405.8431$