Snap back to reality

Since $a+b+c=0$ $=>$ $(a+b)^2=(-c)^2$ ,Hence $a^2+b^2+2 ab=c^2$ . Substituting $a^2+b^2=1-c^2$ ;we get $ab=c^2-\frac{1}{2}$ $=>$ $abc=c^3-\frac{c}{2}$ . Now consider a function $f(c)=c^3-\frac{c}{2}$ ,since maxima occurs when $\frac{dy}{dx}=0$ .Hence maxima will occur at $c=\frac{1}{\sqrt{6}}$ .Substituting value of $c$ [= $\frac{1}{\sqrt{6}}$ ] in the $f(c)$ ,we get $abc=\frac{1}{3\sqrt{6}}$ .Hence $k=\frac{1}{3\sqrt{6}}$ $=>$ $k^2=54$

Without losing generality, we assume $ab\geq 0$

From the condition, we have $a+b=-c$ $\Rightarrow a^2+b^2+(a+b)^2=1 \ (*)$ By Power - Mean we get $a^2+b^2\geq\frac{(a+b)^2}{2}$ $\Rightarrow \frac{3}{2}(a+b)^2\leq 1$ $\Leftrightarrow c^2\leq\frac{2}{3}\Rightarrow c\leq\frac{2}{\sqrt{6}}$ From $(*)$ , we get $2(a^2+b^2+ab)=1\geq 6ab\Rightarrow ab\leq\frac{1}{6}$ $\therefore abc\leq\frac{1}{3\sqrt{6}}\Rightarrow k=3\sqrt{6}\Rightarrow k^2=54$ The equality holds when $(a;b;c)=\bigg(\frac{-1}{\sqrt{6}};\frac{-1}{\sqrt{6}};\frac{2}{\sqrt{6}}\bigg)$

I just substituted $c=-a-b$ in the other equations and went straight for Lagrange Multipliers: $abc=-ab(a+b),$ $a^2+ab+b^2 = \frac{1}{2}.$ Applying the Lagrange Multipliers approach leads to: $-2ab-b^2=\lambda (2a+b),$ $-a^2-2ab=\lambda (2b+a),$ $a^2+ab+b^2=\frac{1}{2}.$ Multiplying the first two equations, I get: $(-2ab-b^2)\lambda (2b+a)=\lambda (2a+b)(-a^2-2ab)$ $\Rightarrow b(2a+b)(2b+a)=a(a+2b)(2a+b).$ This implies $a=b$ or $b=-2a$ or $a=-2b$ .

Substitute these, in turn, into $a^2+ab+b^2=\frac{1}{2}.$

$a=b\Rightarrow a=\pm\frac{1}{\sqrt{6}}, b=\pm\frac{1}{\sqrt{6}}$

$\Rightarrow abc=\mp\frac{1}{\sqrt{54}}.$

$b=-2a \Rightarrow a=\pm\frac{1}{\sqrt{6}}, b=\mp\frac{2}{\sqrt{6}}$

$\Rightarrow abc=\mp\frac{1}{\sqrt{54}}.$

$a=-2b \Rightarrow a=\pm\frac{2}{\sqrt{6}}, b=\mp\frac{1}{\sqrt{6}}$

$\Rightarrow abc = \pm\frac{1}{\sqrt{54}}.$

Because $a$ and $b$ are confined to the ellipse $a^2+ab+b^2=\frac{1}{2}$ , I believe we can conclude that the global maximum is $\frac{1}{\sqrt{54}}$ . (If my reasoning is incorrect, please let me know. Otherwise, please point me to the right theorem or elaborated reasoning to conclude this.)