Find The Maximum

Note that from $AM-GM,$ $(2x+y+z)^2= ((x+y) + (z+x))^2 \geq (2 \sqrt{(x+y)(z+x)} )^2 = 4 (x+y)(z+x).$ Analogously we get $(x+2y+z)^2 \geq 4(x+y)(y+z) \\ (x+y+2z)^2 \geq 4(z+x)(x+y).$ Thus, $\begin{array}{lcl} \dfrac{1}{(2x+y+z)^2} + \dfrac{1}{(x+2y+z)^2} + \dfrac{1}{(x+y+2z)^2} & \leq & \dfrac{1}{4 (x+y)(y+z)} + \dfrac{1}{4 (y+z)(z+x)} + \dfrac{1}{4 (z+x)(x+y)} \\ & = & \dfrac{x+y+z}{2(x+y)(y+z)(z+x)}. \end{array}$ Now, note that by AM-GM, $\begin{array}{lcl} (xy+yz+zx)^2 &= & x^2y^2 + y^2z^2 + z^2x^2 + 2 (xy^2z + yz^2x + x^2yz) \\ & \geq & (xy^2z + yz^2x + x^2yz) \\ & = & 3 (xy^2z + xyz^2 + zx^2y), \end{array}$ so $(xy+yz+zx)^2 \geq 3xyz(x+y+z) .$ It is well known that $(x+y)(y+z)(z+x) \geq \dfrac{8}{9}(x+y+z)(xy+yz+zx).$ (For a proof, use AM-GM on $(x^2y, y^2z, z^2x, xy^2, yz^2, xz^2)$ and then some tedious simplifications.)

Now, $\begin{array}{lcl} \dfrac{1}{(2x+y+z)^2} + \dfrac{1}{(x+2y+z)^2} + \dfrac{1}{(x+y+2z)^2} & \leq & \dfrac{x+y+z}{2(x+y)(y+z)(z+x)} \\ & = & \dfrac{(x+y+z)(xy+yz+zx)}{2(x+y)(y+z)(z+x)} \cdot \dfrac{xy+yz+zx}{xyz(x+y+z)} \cdot \dfrac{xyz(x+y+z)}{(xy+yz+zx)^2}\\ & \leq & \dfrac{9}{16} \times \dfrac{1}{3} = \dfrac{3}{16}. \end{array}$ Equality holds for $(x,y,z)= (1,1,1).$ Thus, $a=b, 3= 16,$ and $a+b= \boxed{19}.$

This problem is adapted from ISL 2009 A2 .