Boxed in a Box

Let the width of the gap be $x$ , as shown.

Then $\Delta ABC$ , is a right angled triangle, with $AB=x+1$ , $BC=x$ and $CA=2$

Applying Pythagoras theorem, $(x+1)^2+x^2=2^2$

$\implies x=\dfrac{\sqrt{7}-1}{2}$

Side length of the larger square $=4+x=\dfrac{\sqrt{7}+7}{2}$

$\text{Shaded area} = \text{Area of the bigger square} - \text{Total area of the smaller squares}$

$=\left(\dfrac{7+\sqrt{7}}{2}\right)^2-18$

$=\boxed{\dfrac{7\sqrt{7}}{2}-4}$

(Edited: changed the origin)

After some analysis, I figured out that any angle $θ$ between 0 and 0.7254440201 will do (outside these bounds, the corner points $A$ through $E$ in the diagram below will no longer be on the edges or same edges of the rotated squares, see my reply below). For reference the diagram given in the question with some points and angles named.

The coordinates of the different points are (given the usual axes: $x$ to the right, $y$ up):

$A=(x, 2+x)\\ B=(1, 0)\\ C=(1+x, 1+x)\\ D=(0, 1)\\ E=(1+x, 1)\\ F=(0, 1+x)$

We then rotate these points by θ counterclockwise to bring them into the coordinate system of the 4 rotated squares. For this, we use rotation matrix $R = \begin{bmatrix}cos(θ) & -sin(θ) \\ sin(θ) & cos(θ)\end{bmatrix}$ .

$A' = R \cdot A = (x \cdot cos(θ) - (2+x) \cdot sin(θ), x \cdot sin(θ) + (2 + x) \cdot cos(θ))\\ B' = R \cdot B = (cos(θ), sin(θ))\\ C' = R \cdot C = ((1+x) \cdot cos(θ) - (1+x) \cdot sin(θ), (1+x) \cdot sin(θ) + (1+x) \cdot cos(θ))\\ D' = R \cdot D = (-sin(θ), cos(θ))\\ E' = R \cdot E = ((1+x) \cdot cos(θ) - sin(θ), (1+x) \cdot sin(θ) + cos(θ))\\ F' = R \cdot F = (-(1+x) \cdot sin(θ), (1+x) \cdot cos(θ))$

We then calculate the vertical distances between the pairs of points that bound the 4 rotated suqares:

$\text{vertical distance } A' \text{ to } D' = x \cdot sin(θ) + (2 + x) \cdot cos(θ) - (cos(θ)) = x \cdot sin(θ) + (1+x) \cdot cos(θ)\\ \text{vertical distance } C' \text{ to } B' = (1+x) \cdot sin(θ) + (1+x) \cdot cos(θ) - (sin(θ)) = x \cdot sin(θ) + (1+x) \cdot cos(θ)\\ \text{horizontal distance } E' \text{ to } F' =(1+x) \cdot cos(θ) - sin(θ) - (-(1+x) \cdot sin(θ)) = (1+x) \cdot cos(θ) + x \cdot sin(θ)$

Each of these distances is actually equal, and should equal 2:

$x \cdot sin(θ) + (1+x) \cdot cos(θ) = 2$

Solving for $x$ gives $x = (2 - cos(θ)) / (sin(θ) + cos(θ))$ .

For every $θ$ between 0 and 0.7254440201, we can find a corresponding gap $x$ . For $θ = 0$ , we find a $x = 1$ , and a blue area of 7 (this is easily verified visually). Finding the $θ$ for which $x$ is minimal, leads to the solution presented by other people (I haven't included this to keep the length down, but I did check). But it is clear that the question needs to specify something to fix the angle, otherwise it is ambiguous.

I don't know if my way has any value because it is based on a very wild guess:

I approximated the blue area. I counted it was approximately 7. Then I guessed that B was the area of the 4 squares in the middle: B = 4.

So I had this equation:

$\frac{A^{(\frac{3}{2})}}{2}-B = 7$

When I solve for A : $A=7.8$ . Thus, the closest prime number is 7 and thus the answer is $7+4 = 11$

Look for and see Mike Jarvis' reply to Peter Vanbroekhoven's solution for the best way to analyze this problem. Length $AC$ is actually arbitrary ----unless we impose the condition that the area of the larger square has to be a minimum. Then $AC$ has to be $2$ . Check Peter Vanbroekhoven's work to see why $AC$ is arbitrary.

The statement of this problem needs to include this additional condition of minimum area of larger square.