Maximize the given function f(x)

Given : $f(x) = \sqrt{x^4-3x^2-6x+13} - \sqrt{x^4-x^2+1}$

$=\sqrt{(x^2-2)^2+(x-3)^2} - \sqrt{(x^2-1)^2+(x-0)^2}$

On the Cartesian Plane if $P(x,x^2) , A(3,2) , B(0,1)$
are three points, then

$y=|PA| - |PB| \le |AB|=\sqrt{10}$ (Using Triangle Inequality)

where the equality holds if $P$ , $A$ and $B$ lie on a line.

$\implies f_{\text{max}} (x) = \boxed{\sqrt{10}}$

Since $f'(x) \; = \; \frac{2x^3 - 3x - 3}{\sqrt{x^4 - 3x^2 - 6x + 13}} - \frac{2x^3-x}{\sqrt{x^4 - x^2 + 1}}$ we see that $\begin{aligned} f'(x) = 0 & \Rightarrow \; (2x^3 - 3x - 3)\sqrt{x^4-x^2+1} \; = \; (2x^3 - x)\sqrt{x^4 - 3x^2 - 6x + 1} \\ & \Rightarrow \; (2x^3 - 3x - 3)^2(x^4 - x^2 + 1) \; = \; (2x^3 - x)^2(x^4 - 3x^2 - 6x + 1) \\ & \Rightarrow \; (3x^2 - x - 3)(4x^5 - 12x^4 + 2x^3 + 3x^2 - 5x - 3) \; = \; 0 \end{aligned}$ There are three real roots of this equation, namely $a_\pm = \tfrac16(1 \pm \sqrt{37})$ and $b \approx 2.79462$ . Of these, while $a_+=\tfrac16(1 + \sqrt{37})$ is a root of the order $7$ polynomial equation, it is not a root of the unsquared version of that equation (since $2a_+^3 - a_+ > 0 > 2a_+^3 - 3a_+ - 3$ ), and so is not a zero of $f'(x)$ . It is easy to check that $a_- = \tfrac16(1-\sqrt{37})$ is a maximum for $f$ , while $b$ is a minimum. Evaluating $f$ at $\tfrac16(1-\sqrt{37})$ gives the maximum value of $\boxed{3.16}$ to 2DP.

---

Alternatively, try this: $\begin{aligned} (3x^2 - x - 3)^2 & \ge \; 0 \\ 10x^4 - 10x^2 + 10 & \ge \; x^4 + 6x^3 + 7x^2 - 6x + 1 \\ 10(x^4 - x^2 + 1) & \ge \; (x^2 + 3x - 1)^2 \\ \sqrt{10}\sqrt{x^4 - x^2 + 1} & \ge \; 1 - 3x - x^2 \\ 2\sqrt{10}\sqrt{x^4 - x^2 + 1} & \ge \; (x^4 - 3x^2 - 6x + 13) - (x^4 - x^2 + 1) - 10 \\ x^4 - 3x^2 - 6x + 13 & \le \; \big(\sqrt{x^4 - x^2 + 1} + \sqrt{10}\big)^2 \\ \sqrt{x^4 - 3x^2 - 6x + 13} & \le \; \sqrt{x^4 - x^2 + 1} + \sqrt{10} \\ f(x) & \le \; \sqrt{10} \end{aligned}$ with equality precisely when $3x^2 - x -3 = 0$ and $x^2 + 3x - 1 < 0$ , namely when $x = \tfrac16(1 - \sqrt{37})$ . It is also worth noting that $x^4 - 3x^2 - 6x + 13 = (x^2 - 2)^2 + (x - 3)^2 \ge 0$ , and hence the square root can always be defined.