Square Into Rectangles

Let the width of the cut-out rectangle be $x$ and the smallest angle between the two rectangles be $\theta$ as shown in the figure above. Then we have:

$\begin{cases} x\cos \theta + \sin \theta = 1 - x & \implies x = \dfrac {1-\sin \theta}{1+\cos \theta} \\ x \sin \theta + \cos \theta = 1 & \implies x = \dfrac {1-\cos \theta}{\sin \theta} \end{cases}$

$\begin{aligned} \implies \frac {1-\sin \theta}{1+\cos \theta} & = \frac {1-\cos \theta}{\sin \theta} \\ \sin \theta - \sin^2 \theta & = 1 - \cos^2 \theta = \sin^2 \theta \\ \sin \theta - 2\sin^2 \theta & = 0 \\ \sin \theta (1-2\sin \theta) & = 0 \\ \implies \sin \theta & = \frac 12 & \small \blue{\text{Since }\theta > 0} \\ \implies \theta & = \frac \pi 6 \end{aligned}$

Therefore $x = \dfrac {1-\frac {\sqrt 3}2}{\frac 12} = 2 - \sqrt 3$ and $a+b = 2+3 = \boxed 5$ .

Label the width of the rectangle $x$ , the short leg of one of the large triangles $y$ , and the other leg $z$ . Since the four triangles are similar and a small triangle has a hypotenuse of $x$ , its other legs are $xy$ and $xz$ .

From the sides of the unit square, $x + xz + y = 1$ and $xy + z = 1$ , and by Pythagorean's Theorem, $y^2 + z^2 = 1$ .

These three equations solve to positive solutions of $x = 2 - \sqrt{3}$ , $y = \frac{1}{2}$ , and $z = \frac{\sqrt{3}}{2}$ , so $a = 2$ , $b = 3$ , and $a + b = \boxed{5}$ .

Let the lower left square corner be $(0,0)$ .

Define some unit vectors:

$u_{1x} = \cos \theta \\ u_{1y} = \sin \theta \\ u_{2x} = -u_{1y} \\ u_{2y} = u_{1x}$

These are the criteria for selecting parameters:

$P_{2x} = 0 \\ P_{4x} = 1 \\ P_{3y} = 1 - w$

Fleshing these out:

$d + w \, u_{2x} = 0 \\ d + u_{1x} = 1 \\ u_{1y} + w \, u_{2y} = 1 - w$

Determine $(w,d, \theta)$ such that the above three equations are satisfied. I solved numerically, yielding:

$w \approx 0.26795 \,\,\, ( 2 - \sqrt{3}) \\ d \approx 0.13397 \\ \theta \approx 30^\circ$

What about limiting case of width =1/2 when rectangles overlap?