That's an Ugly Looking Octagon

The first thing we need to figure out is how do we transform this ugly octagon into plain beauty!

Imagine cutting the octagon into pieces from the centre to the vertices and labelling them from 1 to 8, like this: Now rearrange the pieces so that it becomes: That's much better. Since rearranging the pieces doesn't change its area, we can calculate the area of the original octagon by calculating the area of this new octagon. Notice that the new octagon is simply a square with 4 of its corners cut off, the cut off corners are all congruent right isosceles triangles (to see why, read the supplementary notes below), thus, using a bit of Pythagoras' theorem, denoting $S_\text{a shape}$ as the area of a shape, we can easily figure out that the area of this octagon is $\begin{aligned} &S_\text{square}-4S_\text{right isosceles triangle} \\&=(\sqrt2+2)^2-4\times \frac12 \times \left(\frac{\sqrt2}{2}\right)^2 \\&=6+4\sqrt2-1 \\&=5+4\sqrt2 \end{aligned}$ Hence, $a=5$ , $b=4$ , $c=2$ , $a+b+c=11$ .

Supplementary notes:

This is an explanation of why the figure above is simply a square with 4 of its corners cut off and the cut off corners are congruent right isosceles triangles.

Suppose the octagon is $A'B'C'D'E'F'G'H'$ and the centre of the circle is $O$ , extend $A'B'$ , $C'D'$ , $E'F'$ , $G'H'$ until they meet at $P$ , $Q$ , $R$ and $S$ .

Let $r$ be the radius of $\odot O$ and $\angle A'OH'=\alpha$ , $\angle A'OB'=\beta$

Since $H'O=B'O=r$ , $A'O=C'O=r$ and $A'H'=B'C'=1$ , we have $\triangle A'OH'\cong\triangle B'OC'$ $(SSS)$

Similarly we can also prove that $\triangle A'OH'\cong\triangle D'OE'\cong\triangle F'OG'$ and $\triangle A'OB'\cong\triangle C'OD'\cong\triangle E'OF'\cong\triangle G'OH'$ $\therefore \angle A'OH'=\angle B'OC'=\angle D'OE'=\angle F'OG'=\alpha\\ \angle A'OB'=\angle C'OD'=\angle E'OF'=\angle G'OH'=\beta\\ \therefore 4\alpha+4\beta=360^\circ\\ \therefore \beta=90^\circ-\alpha$ Since $\triangle A'OH'$ and $\triangle A'OB'$ are isosceles, we have $\angle H'A'O=90^\circ-\frac\alpha2$ and $\angle B'A'O=90^\circ-\frac\beta2=45^\circ+\frac\alpha2$ $\therefore\angle H'A'B'=\angle H'A'O+\angle B'A'O=135^\circ$ By similar means we can also prove that all the other internal angles of the octagon are $135^\circ$

In $\triangle H'A'P$ , $\angle PA'H'=\angle PH'A'=45^\circ$ , $\therefore \angle P=90^\circ$ , $\triangle H'A'P$ is a right isosceles triangle.

Similarly we can also prove that $\triangle B'QC'$ , $\triangle D'RE'$ and $\triangle F'SG'$ are all right isosceles triangles, and because $H'A'=B'C'=D'E'=F'G'=1$ , they are all congruent $(ASA)$ .

Finally, $PQ=PA'+A'B'+B'Q=PH'+H'G'+G'S=PS$ , similarly we can prove $PQ=QR=RS=PS$ , and since $\angle P=90^\circ$ , $PQRS$ is a square. We have proved that the octagon is simply a square with 4 right isosceles triangles cut off.

Let O be the centre of the circle and $r$ be the radius of the circle.

The area would be unchanged if the sides were alternating (1, 2, 1, 2, etc). This means $\angle$ EOG = 90 $^\circ$ .

Let $\angle$ GOF = $\theta$ . Then $\angle$ EOF = 90 - $\theta$ .

The cosine rule ( $a^2 = b^2 + c^2 - 2bc\cos A$ ) on GOF and EOF respectively gives:

$4 = 2r^2 - 2r^2\cos \theta$

$1 = 2r^2 - 2r^2\cos (90-\theta)$ $\boxed1$

Noting that $\cos (90-\theta) = \sin \theta$ , rearranging these gives:

$2r^2 = \frac{4}{1-\cos\theta}$

$2r^2 = \frac{1}{1-\sin\theta}$

Setting the RHSs of these equal and multiplying out the denominators gives:

$4-4\sin\theta=1-\cos\theta$

$4\sin\theta-3=\cos\theta$

$4\sin\theta-3=\sqrt{1-\sin^2\theta}$

Squaring both sides leads to the quadratic in $\sin\theta$ :

$17\sin^2\theta -24\sin\theta+8=0$

So $\sin\theta=\frac{12+2\sqrt2}{17}$ $\boxed2$ .

$\sin^2+\cos^2=1$ gives $\cos\theta=\frac{-3+8\sqrt2}{17}$ $\boxed3$ .

Now the area required is equivalent to 4 times the area of OEFG.

Area = $4(\frac12r^2\sin\theta +\frac12r^2\sin(90-\theta))$

Area = $4(\frac12r^2\sin\theta +\frac12r^2\cos\theta)$

Area = $2r^2(\sin\theta +\cos\theta)$

Substituting using equation $\boxed1$ ,

Area = $\frac1{1-\sin\theta}(\sin\theta +\cos\theta)$

Using surd forms (equations $\boxed2$ and $\boxed3$ ) ultimately simplifies to:

Area = $5+4\sqrt2$ .

So $a+b+c = 5+4+2=11$ .

Surely there's a simpler approach!?

my original.
Using the beautiful concept of Tan Kenneth.