Find the maximum value

The maximum value of $x^4 y + xy^4$ is actually $\frac{1}{12}$ , which occurs when $(x,y) = \left( \frac{3 \pm \sqrt{3}}{6}, \frac{3 \mp \sqrt{3}}{6} \right).$

See https://brilliant.org/problems/progessions-and-sequences/ and https://brilliant.org/problems/differentiation-or-inequality/ .

This problem actually has a really nice solution.

$\begin{aligned} x^4y + xy^4 &= xy(x^3 + y^3)\\ &= xy(x+y)(x^2-xy+y^2)\\ &= xy((x+y)^2-3xy)\\ &=xy(1-3xy)\\ &= xy(1-3xy)\\ \end{aligned}$

From AM-GM:

$\begin{aligned} \dfrac{3xy + (1 - 3xy)}{2} &\ge \sqrt{3xy(1-3xy)}\\ \dfrac{1}{2} &\ge \sqrt{3xy(1-3xy)}\\ \dfrac{1}{4} &\ge 3xy(1-3xy)\\ xy(1-3xy) &\le \dfrac{1}{12} \end{aligned}$

Therefore, $x^4y + xy^4 \le \boxed{\dfrac{1}{12}}$ .