What´s The Radius?

Let the center and radius of the red circle be $O$ and $r$ respectively, $M$ be the midpoint of $CD$ and $P$ be on $DA$ such that $OP$ is parallel to $CD$ . Then we note that $AP = R-r$ , $OP = \dfrac R2$ , and $AO = R+r$ . By Pythagorean theorem , we have:

$\begin{aligned} (R-r)^2 + \left(\frac R2\right)^2 & = (R+r)^2 \\ \frac {R^2}4 & = (R+r)^2 - (R-r)^2 = 4Rr \\ \implies r & = \boxed{\frac R{16}} \end{aligned}$

Let the mid point of AB be E, r be the radius and F be the centre of the red circle. Then AF = R+r, AE = $\dfrac{R}{2}$ , and EF = R - r. Since the angle AEF is a right angle, therefore $(R+r)^2-(R-r) ^2$ = $\dfrac{R^2}{4}$ . Or r= $\dfrac{R}{16}$

If we draw a vertical line with the radius of the red circle until the side of the square, we can get a formula for this problem We can divide that line such that the sum of all these segments equals to $R$ , like this: The radius of the circle $(2r)$ added with the lines a and b equal to $R$ :

$2r+a+b=R..................(1)$

We can calculate $b$ with the intersection point between $\stackrel{\textstyle\frown}{\mathrm{BD}}$ and $\stackrel{\textstyle\frown}{\mathrm{CA}}$ : Suppose that point $A$ has coordinates $(0,0)$ , now let´s find the equations of both circumferences.The center of $\stackrel{\textstyle\frown}{\mathrm{BD}}$ is $(0,0)$ , so its equation is $y=\sqrt{R^{2}-x^{2}}$ and the center of $\stackrel{\textstyle\frown}{\mathrm{CA}}$ is $(R,0)$ , so its equation is $y=\sqrt{R^{2}-(x-R)^{2}}$ . The intersection can be found with the equation

$\sqrt{R^{2}-x^{2}}=\sqrt{R^{2}-(x-R)^{2}}$

Solving this, the intersection point is $(\frac{R}{2},\frac{R\sqrt{3}}{2})$ . In this case $b$ is the y-coordinate of the intersection point.

Now, for $a$ , we can draw a triangle with the center point of the red circle and points $A, B$ . We can create a right triangle dividing the side of the square by 2, with lengths like the graph above. Use Pythagoras´ theorem to find $a$ .

$(\frac{R}{2})^{2}+(r+\frac{R\sqrt{3}}{2}+a)^{2}=(R+r)^{2}$

We can calculate $r$ in terms of $R$ and $a$ using the first equation

$R=2r+a+b=2r+a+\frac{R\sqrt{3}}{2}$ , with this, $r=\frac{R(2-\sqrt{3})-2a}{4}......................................(2)$

Therefore, the new equation is

$(\frac{R}{2})^{2}+(\frac{R(2+\sqrt{3})+2a}{4})^{2}=(\frac{R(6-\sqrt{3})-2a}{4})^{2}$

Solving this equation for $a$ , $a=\frac{R(7-4\sqrt{3})}{8}$ .

Finally, substitute $a$ in the second equation

$r=\frac{R(2-\sqrt{3})-2(\frac{R(7-4\sqrt{3})}{8})}{4}$

$r=\frac{R}{16}$