Rectangle in a quarter circle

Let the rectangle be $ABCD$ , with vertex $A$ on the $x$ -axis and side lengths $a$ and $3a$ , $O$ be the origin of the $xy$ -plane, $BN$ be perpendicular to the $x$ -axis, and $M$ be the midpoint of $BC$ . Due to symmetry, we know that $OM \parallel AB$ and $\angle MON = \angle BAN = 45^\circ$ . By Pythagorean theorem ,

$\begin{aligned} ON^2 + BN^2 & = OB^2 \\ (OA+AN)^2 + BN^2 & = OB^2 \\ \left(\frac 32a\sqrt 2 + \frac a{\sqrt 2}\right)^2 + \left(\frac a{\sqrt 2}\right)^2 & = 1^2 \\ 8a^2 + \frac {a^2}2 & = 1 \\ \implies a^2 & = \frac 2{17} \end{aligned}$

The area of $ABCD$ , $[ABCD]=AB \times BC = 3a^2 = \boxed{\frac 6{17}}$ .

It's not difficult to see that if the rectangular vertex touching x-axis has a co-ordinate of (3m, 0), then the rightmost one would be (4m, m). Then,
R² = (4m)² + m² = 17m² = 1² = 1.

17 only appears once in the options, so 6/17 it is.

[ ] (https://brilliant.org)