A cone, a plane, and two spheres

Clearly, the center of both spheres is on the positive z-axis. Let the center be $(0, 0, z)$ , then the radius is $r = z \sin( \dfrac{\pi}{6} ) = z/2$ . In addition the distance from the center of a sphere to the plane is also equal to $r$ . Thus, for the lower sphere, we have,

$r = \dfrac{(5 - 4 z)}{\sqrt{ 2^2 + 3^2 + 4^2 }} = \dfrac{(5 - 4 z)}{ \sqrt{29}} = \dfrac{z}{2}$

Solving for z we get,

$z_1 = \dfrac{ 5 / \sqrt{29} } { \dfrac{1}{2} + \dfrac{4}{\sqrt{29} } } = \dfrac{ 5 } { \dfrac{1}{2} \sqrt{29} + 4 }$

and therefore,

$r_1 = \dfrac{ 5 } { \sqrt{29} + 8 }$

And for the bigger (upper sphere) , we have

$r = \dfrac{(4 z - 5)}{ \sqrt{29}} = \dfrac{z}{2}$

Solving for z,

$z_2 = \dfrac{ 5 / \sqrt{29} } { \dfrac{4}{\sqrt{29}} - \dfrac{1}{2} } = \dfrac{ 5 } { 4 - \dfrac{1}{2} \sqrt{29} }$

hence,

$r_2 = \dfrac{z_2}{2} = \dfrac{5}{8 - \sqrt{29} }$

Therefore, the required ratio is

$\dfrac{r_2}{r_1} = \dfrac{ \sqrt{29} + 8 } { 8 - \sqrt{29} } = 5.11893248326$

Let one of the spheres have a center at $S$ , a radius of $r$ , and tangent to point $T$ on the plane $2x + 3y + 4z = 5$ .

Since the cone opens upward with an apex at the origin and has a semi-vertical angle of $\frac{\pi}{6}$ , $S$ has coordinates $(0, 0, \frac{r}{\sin \frac{\pi}{6}}) = (0, 0, 2r)$ .

Since the sphere is tangent to point $T$ on the plane $2x + 3y + 4z = 5$ , $\vec{ST} = (2k, 3k, 4k)$ for some $k$ , and since its length is the radius of the sphere $r$ , $\sqrt{(2k)^2 + (3k^2) + (4k^2)} = r$ , which solves to $k = \pm \frac{r}{\sqrt{29}}$ , and therefore $\vec{ST} = (\pm \frac{2r}{\sqrt{29}}, \pm \frac{3r}{\sqrt{29}}, \pm \frac{4r}{\sqrt{29}})$ .

Since $S$ has coordinates $(0, 0, 2r)$ , and $\vec{ST} = (\pm \frac{2r}{\sqrt{29}}, \pm \frac{3r}{\sqrt{29}}, \pm \frac{4r}{\sqrt{29}})$ , $T$ has coordinates $(0 \pm \frac{2r}{\sqrt{29}}, 0 \pm \frac{3r}{\sqrt{29}}, 2r \pm \frac{4r}{\sqrt{29}}) = (\pm \frac{2r}{\sqrt{29}}, \pm \frac{3r}{\sqrt{29}}, 2r \pm \frac{4r}{\sqrt{29}})$ .

But since $T$ is also on the plane $2x + 3y + 4z = 5$ , we obtain the equation $2(\pm \frac{2r}{\sqrt{29}}) + 3(\pm \frac{3r}{\sqrt{29}}) + 2(2r \pm \frac{4r}{\sqrt{29}})) = 5$ , which solves to $r = \frac{5}{8 \pm \sqrt{29}}$ .

The ratio of the radius of the bigger sphere to the radius of the smaller sphere is then $\frac{\frac{5}{8 - \sqrt{29}}}{\frac{5}{8 + \sqrt{29}}} = \frac{8 + \sqrt{29}}{8 - \sqrt{29}} \approx \boxed{5.11893248326}$ .