HexaRatio

Let each side of the regular hexagon be $s$ , so that its perimeter is $P_{ABCDEF} = 6s$ , and reflect $\triangle MBC$ along $MC$ and $\triangle CDN$ along $CN$ :

Since $\angle BCM + \angle NCD = \angle BCD - \angle MCN = 120° - 90° = 30°$ , and $\angle BCM = \angle B'CM$ and $\angle NCD = \angle NCD'$ , then $\angle B'CM + \angle NCD' = \angle BCM + \angle NCD = 30°$ , so $\angle B'CD' = \angle MCN - (\angle B'CM + \angle NCD') = 90° - 30° = 60°$ .

Since $BC = B'C = CD = CD' = s$ , and since $\angle B'CD' = 60°$ , $\triangle B'CD'$ is an equilateral triangle, which means $B'D' = s$ .

Also, $AM + MB' = AM + MB = s$ , and $D'N + NE = DN + NE = s$ .

That means the perimeter of $AMNEF$ is $P_{AMNEF} = (AM + MB') + B'D' + (D'N + NE) + EF + AF = s + s + s + s + s = 5s$ .

The ratio of the perimeters is then $\cfrac{P_{AMNEF}}{P_{ABCDEF}} = \cfrac{5s}{6s} = \cfrac{5}{6}$ .

Therefore, $a = 5$ , $b = 6$ , and $a + b = \boxed{11}$ .

My approach was based on 3D coordinate geometry. Why 3D? Because this enables us to simplify the numbers based on the fact that, if you cut a cube in half perpendicularly to a vertex axis, the cross-section so produced is a regular hexagon.

So let the coordinates be as follows: $A = ( 0, -2, -2) \\ B = (-2, 0, -2) \\ C = (-2, 2, 0) \\ D = ( 0, 2, 2) \\ E = ( 2, 0, 2) \\ F = ( 2, -2, 0) \\ M = (-1, -1, -2) \\ N = (t, 2-t, 2)$ The dot product of non-zero vectors is zero if and only if they are perpendicular. Thus we can solve: $\overrightarrow{CM} . \overrightarrow{CN} = 0 \\ \Rightarrow \begin{pmatrix} 1 \\ -3 \\ -2 \end{pmatrix} . \begin{pmatrix} t+2 \\ -t \\ 2 \end{pmatrix} = 0 \\ \Rightarrow (1) (t+2) + (-3)(-t) + (-2) (2) = 0 \\ \Rightarrow 4t - 2 = 0 \\ \Rightarrow t = \frac12$ This gives $N = \left(\frac12, \frac32, 2\right).$

Now we can calculate the perimeters of the two shapes. Perimeter of hexagon: $h = 6 AF \\ = 6 \sqrt8 \\ = 12 \sqrt2$ Perimeter of pentagon: $p = AM + MN + NE + EF + FA \\ = \sqrt2 + \frac72 \sqrt2 + \frac32 \sqrt2 + 2 (2 \sqrt2) \\ = 10 \sqrt2$ Ratio: $\displaystyle{\frac{h}{p} = \frac{12\sqrt2}{10\sqrt2} = \boxed{\frac65}}.$

The unit hexagon can be presented in the lattice points of unit equilateral triangle as shown above. Notice that by extending the leg perpendicular to one of the segments, we have a right green triangle with leg lengths $\dfrac{11}{2}$ and $\dfrac{5}{2}\sqrt{3}$ . Following Pythagorean’s Theorem, $\sqrt{\left(\dfrac{11}{2}\right)^2+\left(\dfrac{5}{2}\sqrt{3}\right)^2} = 7$ . Therefore,

$\dfrac{\text{Perimeter of 5-gon}}{\text{Perimeter of 6-gon}} = \dfrac{2+3+4+4+7}{4 \times 6} = \dfrac{5}{6}$

which returns us $a + b = 5 + 6 = \boxed{11}$ .

Note that if we extend the segment as shown above, it is easier to see that the right triangle of side lengths $1$ and $\dfrac{\sqrt{3}}{2} \times 8 = 4\sqrt{3} = \sqrt{48}$ has the hypotenuse $\sqrt{49} = 7$ .

For simplicity we scale $ABCDEF$ to a unit hexagon. Then $AC$ is the side of an equilateral triangle inscribed in the hexagon’s (unit) circumsircle, thus, $AC=\sqrt{3}$ . Let $L$ be the midpoint of $DE$ and ${{N}_{1}}$ the midpoint of $DL$ .

First we prove that ${{N}_{1}}$ in fact is the given point $N$ .
By Apollonius's Theorem on $\triangle ABC$ , $C{{B}^{2}}+C{{A}^{2}}=2\left\{ C{{M}^{2}}+{{\left( \frac{AB}{2} \right)}^{2}} \right\}$ from which we get $C{{M}^{2}}=\dfrac{2C{{B}^{2}}+2C{{A}^{2}}-A{{B}^{2}}}{4}=\dfrac{2\times {{1}^{2}}+2\times {{\sqrt{3}}^{2}}-{{1}^{2}}}{4}=\dfrac{7}{4}$ Similarly, for the median $C{{N}_{1}}$ of $\triangle CDL$ , $C{{N}_{1}}^{2}=\dfrac{2C{{D}^{2}}+2C{{L}^{2}}-D{{L}^{2}}}{4}=\dfrac{2C{{D}^{2}}+2C{{M}^{2}}-D{{L}^{2}}}{4}=\dfrac{2\times {{1}^{2}}+2\times \dfrac{7}{4}-{{\left( \dfrac{1}{2} \right)}^{2}}}{4}=\dfrac{21}{16}$ By Pythagorean theorem on right $\triangle ML{{N}_{1}}$ , $M{{N}_{1}}^{2}=M{{L}^{2}}+L{{N}_{1}}^{2}={{\left( 2OL \right)}^{2}}+{{\left( \dfrac{DL}{2} \right)}^{2}}={{\left( 2\cdot \dfrac{\sqrt{3}}{2} \right)}^{2}}+{{\left( \dfrac{1}{4} \right)}^{2}}=\dfrac{49}{16}$ thus $M{{N}_{1}}=\dfrac{7}{4}$ We notice that $C{{M}^{2}}+C{{N}_{1}}^{2}=\dfrac{7}{4}+\dfrac{21}{16}=\dfrac{49}{16}=M{{N}_{1}}^{2}$ Hence, by the converse of Pythagorean theorem ( Euclid's Elements, Book I, Proposition 48 ), $\angle MC{{N}_{1}}=90{}^\circ$ and this proves that ${{N}_{1}}$ coincides with given point $N$ .

The perimeter ${{P}_{5}}$ of pentagon $AMNEF$ is ${{P}_{5}}=AM+MN+NE+EF+FA=\frac{1}{2}+\frac{7}{4}+\frac{3}{4}++1+1=5$ and the perimeter ${{P}_{6}}$ of the regular hexagon is 6, hence, $\dfrac{{{P}_{5}}}{{{P}_{6}}}=\dfrac{5}{6}$ For the answer, $a=5$ , $b=6$ , thus, $a+b=\boxed{11}$ .

Original (trig) solution in a comment below.

Say the hexagon has unit sides. Note that $AB$ and $DE$ are parallel. To make things a bit easier to see, rotate the image so that these lines are horizontal:

Point $D'$ lies on the extension of line $DE$ so that $DD'C$ is a right-angle; point $B'$ is defined similarly. $DD'=BB'=\frac12$ and $B'C=CD'=\frac{\sqrt3}{2}$ .

Since the triangles $B'CM$ and $D'CN$ are similar, $\frac{D'N}{B'C}=\frac{CD'}{B'M}$

so $B'M \cdot D'N = B'C \cdot CD' = \frac34$ .

By Pythagoras... $\begin{aligned} MN^2&=CM^2+CN^2 \\ &=\left(B'C^2+B'M^2 \right)+\left(CD'^2+D'N^2 \right) \\ &=\frac32+B'M^2+D'N^2 \\ &= 2B'M\cdot D'N+B'M^2+D'N^2 \\ &=(B'M+D'N)^2 \\ &=(1+BM+DN)^2 \end{aligned}$

so $MN=1+BM+DN=1+(1-AM)+(1-EN)=3-AM-EN$ .

Hence the perimeter of $AMNEF$ is $AM+MN+NE+EF+FA=AM+(3-AM-NE)+NE+1+1=5$

so the ratio is $\frac{\text{Perimeter 5-gon}}{\text{Perimeter 6-gon}}=\boxed{\frac56}$

...which doesn't depend on the position of $M$ along $AB$ !