Equilateral triangles in a circle

Let the radius of the circle be 1 and its center be $O$ . The center of the circumcircle is also the centroid of the large triangle. Then the ratio of lengths from vertex and base to centroid is $2:1$ , therefore $OP= \frac 12$ . The side length of the inscribed large triangle is $\sqrt 3$ . Note that $\angle OPQ = 150^\circ$ . Let the side length of the small triangle be $a$ . By cosine rule, we have:

$\begin{aligned} OQ^2 & = OP^2+PQ^2 - 2OP\cdot PQ \cos \angle OPQ \\ 1 & = \frac 14 + a^2 - 2\left(\frac 12\right) a \cos 150^\circ \\ 1 & = \frac 14 + a^2 + 2\cdot \frac 12 \cdot \frac {\sqrt 3}2 a \end{aligned}$

$\implies 4a^2 + 2\sqrt 3 a - 3 = 0$

$\begin{aligned} \implies a & = \frac {-2\sqrt 3 + \sqrt{12+48}}8 & \small \color{#3D99F6} \text{Note that }a> 0 \\ & = \frac {\sqrt {15}-\sqrt 3}4 \end{aligned}$

The ratio of the side lengths of the large and small triangles is $\dfrac {\sqrt 3}a = \dfrac {4\sqrt 3}{\sqrt{15}-\sqrt 3} \approx \boxed{3.236}$ .

Let $O$ be the centre of the circle of radius $r$ , $P$ be the upper vertex and $Q$ the lower left vertex of the small triangle. Then $\Delta OPQ$ is such that $|OQ| = r, |OP| = \dfrac{r}{2}$ and $\angle OPQ = 150^{\circ}$ . So letting the side length $PQ$ of the small triangle be $x$ , by the cosine rule we have that

$|OQ|^{2} = |OP|^{2} + |PQ|^{2} - 2|OP||PQ|\cos(150^{\circ}) \Longrightarrow r^{2} = \left( \dfrac{r}{2} \right)^{2} + x^{2} - rx\cos(150^{\circ}) \Longrightarrow x^{2} - \dfrac{\sqrt{3}r}{2} x - \dfrac{3r^{2}}{4} = 0$ .

The positive solution of this last quadratic in $x$ is $x = \dfrac{\sqrt{3}(\sqrt{5} - 1)r}{4}$ . The side length of the big triangle, given that it's altitude is $\dfrac{3r}{2}$ , is $\sqrt{3}r$ , so the desired ratio is

$\dfrac{\sqrt{3}r}{\dfrac{\sqrt{3}(\sqrt{5} - 1)r}{4}} = \dfrac{4}{\sqrt{5} - 1} = \sqrt{5} + 1 \approx \boxed{3.236}$ .