Find the largest Circle

Let the inradius be $r$ and $\angle ACO = \theta$ . Then $\angle AOC = 180^\circ - 2\theta$ and:

$\begin{aligned} r \cot \frac {\angle AOC}2 + r \cot \frac {\angle ACO}2 & = OC \\ r \cot (90^\circ - \theta) + r \cot \frac \theta 2 & = 1 \\ r \tan \theta + r \cot \frac \theta 2 & = 1 & \small \blue{\text{Let }t = \tan \frac \theta 2} \\ \frac {2t}{1-t^2} r + \frac rt & = 1 \\ \frac {1+t^2}{t(1-t^2)} r & = 1 \\ \implies r & = \frac {t(1-t^2)}{1+t^2} & \small \blue{\text{To find maximum }r} \\ \frac {dr}{dt} & = \frac {(1-3t^2)(1+t^2) - 2t^2(1-t^2)}{(1+t^2)^2} \\ & = \frac {1-4t^2-t^4}{(1+t^2)} & \small \blue{\text{Putting }\frac {dr}{dt} = 0} \\ t^4+4t^2 - 1 & = 0 \\ (t^2+2)^2 & = 5 \\ t^2 & = \sqrt 5 - 2 \\ \implies t & = \sqrt{\sqrt 5 -2} & \small \blue{\text{Since }\tan \frac \theta 2 > 0} \\ \implies \max (r(t)) & = r \left(\sqrt{\sqrt 5-2}\right) & \small \blue{\text{As }r \ge 0} \\ & = \frac {\sqrt{\sqrt 5 - 2}(3-\sqrt 5)}{\sqrt 5 - 1} \\ & = \frac {\sqrt{\sqrt 5 - 2}(2\sqrt 5-2)}4 \\ & = \frac {\sqrt{(\sqrt 5 - 2)(6-2\sqrt 5)}}2 \\ & = \sqrt{\frac {5\sqrt 5 -11}2} \end{aligned}$

Therefore $a+b+c = 5+11 + 2 = \boxed {18}$ .

Let $\theta = \measuredangle AOB$ . The area of the isosceles triangle $ABC$ is $\mathcal A := \frac12 \cdot 1 \cdot 1 \cdot \sin \theta = \frac12 \sin \theta \tag1$

By cosine rule , the length $AC$ satisfies $(AC)^2 = 1^2 + 1^2 - 2\cdot 1\cdot 1 \cdot \cos \theta = 2 (\underbrace{1 - \cos\theta}_{=\, 2\sin^2(\tfrac\theta2) }) \quad \Leftrightarrow \quad AC = 2\sin(\tfrac\theta2) \tag2$

Let the radius of the incircle be denote by $r$ , then $\mathcal A = r \cdot s$ , where $s$ is the semiperimeter of the triangle $AOC$ , $s = \frac{AO + OC + AC}2 = 1 + \sin(\tfrac\theta2) \tag 3$

Thus, we want to maximize $r= \frac{\mathcal A}s = \frac12 \cdot \frac{\sin \theta}{1 + \sin \tfrac\theta2},\quad\quad\quad\quad\quad\quad 0< \theta <\pi \tag4$

Note that as $\theta$ approaches 0 or $\pi$ , $r$ approaches 0. So one of the critical point(s) of $r$ must be a maximum value.

At its critical point(s), $\dfrac{dr}{d\theta} = 0$ . Applying quotient rule produces $2 \Big [ \sin \left( \tfrac \theta 2 \right) + 1 \Big ] \cdot \underbrace{\cos \theta}_{=\, 2 \sin^2(\tfrac\theta2)} - \underbrace{\sin \theta}_{=\, 2 \sin(\tfrac\theta2) \cos(\tfrac\theta2)} \cdot \cos\left( \tfrac\theta 2 \right) = 0$

For simplicity sake, let $y = \sin(\tfrac\theta2) > 0$ , then $\cos^2 (\tfrac\theta2) = 1 - y^2$ , the equation above becomes $2(y + 1)\cdot 2y^2 - 2y \cdot (1-y^2) = 0 \quad \Leftrightarrow \quad y^3 + 2y^2 - 1 = 0\quad \Leftrightarrow \quad (y + 1)(y^2 + y - 1) = 0 \quad \Leftrightarrow \quad y = \dfrac{-1+\sqrt5}2 \text{ only}$

We're left to substitute $\sin \left (\dfrac\theta2\right) = \dfrac{-1 + \sqrt5}2$ into $(4)$ .

Rewriting $\sin \theta = 2\sin(\tfrac\theta2) \cos( \tfrac\theta2) = 2\sin(\tfrac\theta2) \cdot \sqrt{ 1 - \sin^2(\tfrac\theta2)},$

we have $\max(r) = \sqrt{\dfrac{5\sqrt5 - 11}2}$ . The answer is $a + b + c = 5 + 11 + 2 = \boxed{18}.$