Which is larger?

For ease of calculation, let the side-length of the equilateral triangle be $6$ .

---

The blue circle is the incircle of $\Delta BEF$ . We know two sides and the included angle for this triangle; so it's fully defined. Specifically, we know $BE=4,\;\;\angle FBE=60^\circ,\;\; BF=3$

There are lots of way to work out the inradius; I'll use $rs=\Delta$ , where $r$ is the inradius, $s$ is the semi-perimeter and $\Delta$ is the area of the triangle.

To find $s$ , we need to know $EF$ . Using the cosine rule, $EF^2=BE^2+BF^2-2BE\cdot BF \cdot \cos \angle FBE = 4^2+3^2-2\cdot 4\cdot 3\cdot \frac12=13$

so $EF=\sqrt{13}$ , and $s=\frac12 (7+\sqrt{13})$ .

The area is $\Delta=\frac12 BE \cdot BF \sin \angle FBE=\frac12 \cdot 4 \cdot 3 \cdot \frac{\sqrt3}{2}=3\sqrt3$

so we find the radius of the blue circle is $\blue{\frac{6\sqrt3}{7+\sqrt{13}} \approx 0.98}$

---

We have to treat the red circle differently. Extend the lines $FE$ and $AC$ to meet at a point $P$ . The red circle is an excircle of the triangle $CEP$ .

Note that $\angle CEP = \angle BEF$ , which we know from the previous triangle. We also know $\angle ECP = 120^\circ$ and $CE=2$ , so now this triangle is fully defined.

Using $r',s',\Delta'$ for the inradius, semi=perimeter and area of triangle $CFP$ , and $R$ for the radius of the red circle, we have $R=\frac{\Delta'}{s'-CE}$

This solution is already getting a bit long, and most of this is standard manipulation now, so in summary:

• by the sine rule in $\Delta BEF$ , we find $\angle CEP = \angle BEF=\sin^{-1} \frac{3\sqrt3}{2\sqrt{13}}$

• using the sine rule in $\Delta CEP$ we get $\Delta'=3\sqrt3$ and $s'=4+\sqrt{13}$

so the radius of the red circle is $\red{\frac{3\sqrt3}{2+\sqrt{13}} \approx 0.93}$

---

Comparing these results, we see the blue circle is larger, but not by much.

The ratio of the radius of the blue circle to the radius of the red circle is $\frac{4+2\sqrt{13}}{7+\sqrt{13}}=\frac{1}{18} (1+5\sqrt{13})$

Let the side-length of the equilateral triangle be $1$ . Then we have $AF=FB=\frac{1}{2}$ , $CE=\frac{1}{3}$ , $EB=\frac{2}{3}$ .
By cosine rule on $\triangle EFB$ we get $E{{F}^{2}}=B{{E}^{2}}+B{{F}^{2}}-2BE\cdot BF\cdot \cos \angle FBE={{\left( \frac{2}{3} \right)}^{2}}+{{\left( \frac{1}{2} \right)}^{2}}-2\cdot \frac{2}{3}\cdot \frac{1}{2}\cdot \cos 60{}^\circ =\frac{13}{36}\Rightarrow EF=\frac{\sqrt{13}}{6}$

---

For the calculation of the radius $r$ of the blue circle we proceed as follows:

$\begin{aligned} & \left[ EBF \right]=r\cdot s {\color{#20A900}{\text{ }\left( \text{where }s\text{ is the semiperimeter of }\triangle EBF \right)}} \\ & \Rightarrow \frac{1}{2}\cdot BF\cdot BE\cdot \sin \angle EBF=r\cdot \frac{1}{2}\left( \frac{1}{2}+\frac{2}{3}+\frac{\sqrt{13}}{6} \right) \\ & \Rightarrow \frac{1}{2}\cdot \frac{1}{2}\cdot \frac{2}{3}\cdot \sin 60{}^\circ =r\cdot \frac{7+\sqrt{13}}{12} \\ & \Rightarrow {\color{#3D99F6}{r=\frac{\sqrt{3}}{7+\sqrt{13}}}} \\ \end{aligned}$

---

Now, let’s see two different ways to find the radius $R$ of the red circle:

1st way (already mentioned by @Fletcher Mattox ) Consider point $D$ on $AC$ , such that $AD=2\cdot DC$ . Then, by symmetry $DF$ is tangent to the red circle and consequently the kite $CEFD$ is tangential to the red circle.

Thus, $\left[ CEFD \right]=\frac{1}{2}\cdot \left( perimeter \right)\cdot \left( inradius \right)=\frac{1}{2}\cdot 2\cdot \left( \frac{1}{3}+\frac{\sqrt{13}}{6} \right)\cdot R=\frac{2+\sqrt{13}}{6}\cdot R \ \ \ \ \ (1)$ On the other hand,

$\left[ CEFD \right]=\dfrac{1}{2}\cdot CF\cdot DE=\dfrac{1}{2}\cdot \dfrac{\sqrt{3}}{2}\cdot \dfrac{1}{3}=\dfrac{\sqrt{3}}{12} \ \ \ \ \ (2)$ Combining $(1)$ and $(2)$ ,

$\frac{2+\sqrt{13}}{6}\cdot R=\frac{\sqrt{3}}{12}\Leftrightarrow {\color{#D61F06}{R=\frac{\sqrt{3}}{4+2\sqrt{13}}}}$

2nd way
Let $P$ be the intersection of lines $AC$ and $FE$ . Then, the red circle is an excircle of $\triangle PCE$ . Denoting by ${s}'$ the semiperimeter of $\triangle PCE$ , we have $\left[ PCE \right]=R\cdot \left( {s}'-CE \right) \ \ \ \ \ (3)$ Line $PF$ is a transversal of $\triangle ABC$ , hence, by Menelaus’ theorem we have

$\dfrac{PA}{PC}\cdot \dfrac{FB}{FA}\cdot \dfrac{EC}{EB}=1\Rightarrow \dfrac{PA}{PC}\cdot 1\cdot \dfrac{1}{2}=1\Rightarrow PA=2\cdot PC\Rightarrow PC=CA=1$ Therefore,

$\left[ PCE \right]=\dfrac{1}{2}\cdot PC\cdot CE\cdot \sin \angle PCE=\dfrac{1}{2}\cdot 1\cdot \dfrac{1}{3}\cdot \sin 120{}^\circ =\dfrac{\sqrt{3}}{12}$ By cosine rule on $\triangle PCE$ we have

$P{{E}^{2}}=C{{E}^{2}}+P{{C}^{2}}-2\cdot CE\cdot PC\cdot \cos \angle PCE={{\left( \dfrac{1}{3} \right)}^{2}}+{{1}^{2}}-2\cdot \dfrac{1}{3}\cdot \cos 120{}^\circ =\dfrac{13}{9}$ Hence, $PE=\dfrac{\sqrt{13}}{3}$ and ${s}'-CE=\dfrac{1}{2}\cdot \left( 1+\dfrac{1}{3}+\dfrac{\sqrt{13}}{3} \right)-\dfrac{1}{3}=\dfrac{2+\sqrt{13}}{6}$ .

Substituting in $(3)$ we get $\dfrac{\sqrt{3}}{12}=R\cdot \dfrac{2+\sqrt{13}}{6}\Rightarrow {\color{#D61F06}{R=\dfrac{\sqrt{3}}{4+2\sqrt{13}}}}$

---

To conclude, since ${\color{#3D99F6}{r=\dfrac{\sqrt{3}}{7+\sqrt{13}}}\approx 0.1633}$ and ${\color{#D61F06}{R=\dfrac{\sqrt{3}}{4+2\sqrt{13}}}\approx 0.1545}$ , the $\boxed{blue}$ circle is larger.

Bonus: The ratio of the radius of the blue circle to the radius of the red circle is

$\dfrac{\frac{\sqrt{3}}{7+\sqrt{13}}}{\frac{\sqrt{3}}{4+2\sqrt{13}}}=\dfrac{4+2\sqrt{13}}{7+\sqrt{13}}=\dfrac{\left( 4+2\sqrt{13} \right)\cdot \left( 7-\sqrt{13} \right)}{\left( 7+\sqrt{13} \right)\cdot \left( 7-\sqrt{13} \right)}=\dfrac{10\sqrt{13}+2}{36}=\dfrac{5\sqrt{13}+1}{18}$ .
Thus, $a+b+c+d=5+13+1+18=\boxed{37}$ .

Let the side length of equilateral $\triangle ABC$ be $1$ . Then $EB=\frac 23$ and $FB=\frac 12$ . Let $F$ be the origin $(0,0)$ of the $xy$ -plane, then the coordinates of $E$ are $\left(\frac 16, \frac 1{\sqrt 3}\right)$ . And $EF = \sqrt{\frac 1{36}+\frac 13} = \frac {\sqrt{13}}6$ .

Let the radius of the blue circle be $r_1$ . Since it is the inradius of $\triangle EFB$ , then the area of the $\triangle EFB$ is given by:

$\begin{aligned} [EFB] & = \frac {(EB+FB+EF)r_1}2 \\ \frac 12 \cdot EB \cdot FE \cdot \sin 60^\circ & = \frac {(EB+FB+EF)r_1}2 \\ EB \cdot FE \cdot \sin 60^\circ & = (EB+FB+EF)r_1 \\ \implies r_1 & = \frac {\frac 23 \cdot \frac 12 \cdot \frac {\sqrt 3}2}{\frac 23+\frac 12+\frac {\sqrt{13}}6} =\frac {\sqrt 3}{7+\sqrt{13}} \approx 0.1633 \end{aligned}$

Let the center of the red circle be $O$ . We note that $O$ lie on $AF$ or the $y$ -axis. Let $O$ be $(0,h)$ , the radius of the red circle be $r_2$ , and $\angle CFE = \theta$ . Then we note that $r_2 = h \sin \theta$ and $r_2 = \left(\frac {\sqrt 3}2-h\right)\sin 30^\circ = \frac 12 \left(\frac {\sqrt 3}2-h\right) \implies h = \frac {\sqrt 3}2-2r_2$ . Then we have:

$\begin{aligned} r_2 & = \left(\frac {\sqrt 3}2-2r_2\right)\blue{\sin \theta} & \small \blue{\text{Note that }\tan \theta = \frac {\sqrt 3}6} \\ & = \frac {\sqrt 3-4r_2}{2\blue{\sqrt {13}}} & \small \blue{\implies \sin \theta = \frac 1{\sqrt{13}}} \\ \implies r_2 & = \frac {\sqrt 3}{4+2\sqrt{13}} \approx 0.1545 \end{aligned}$

Therefore $r_1 > r_2$ ; the blue circle is larger.

Bonus :

$\frac {r_1}{r_2} = \frac {4+2\sqrt{13}}{7+\sqrt{13}} = \frac {(4+2\sqrt{13})(7-\sqrt{13})}{(7+\sqrt{13})(7-\sqrt{13})} = \frac {2+10\sqrt{13}}{36} = \frac {1+5\sqrt{13}}{18}$

Lets look at the red circle first. If line CE is less than AF, and the red circle is closer to line CE, then the diameter of the red circle is less than half of the length of one side of the triangle.

Because line FB is half the length of one of the sides of the circle, and the blue circle is near line FB; the length of the diameter of the blue circle is exaxtly half of the length of the equalateral triangle.

We know the length of a circle's diameter can deterime the length of the entire circle because the radius of the circle is $\frac{1}{2}$ diameter, and the area of a circle is π ⋅ radius squared, which means the diameter of the circle can also determine the area.

Obviously, half is greater than less than half (assuming we're revering to the length of one side of the equalateral triangle, where the length never changes), so the blue circle is greater than the red circle.