Think Geometry, Not Algebra

The first thing we should notice is that if we look under the radical and sum up the two polynomials being multiplied together, we get $2x^{2}-14x+2y^{2}-14y+74$ . This hints towards being able to factor this expression, so let's begin with that.

For the radical, we can factor that down into $2\sqrt{((x-4)^{2}+y^{2})((x-3)^{2}+(y-7)^{2})}$

As for the rest of the expression, that can be grouped into $y^{2}+(y-7)^{2}+(x-3)^{2}+(x-4)^{2}$

And so we can factor the entire expression into $(\sqrt{(x-3)^{2}+(y-7)^{2}}+\sqrt{(x-3)^{2}+y^{2}})^{2}$ .

And now we see what the problem is really asking. We are looking to minimize the square of the sum of the distances from points $A(3,7)$ and $B(4,0)$ to a point $(x,y)$ . Our constraint $3x+y-6=0$ gives us our points to choose from. Call this line $L$ .

Let's first start by figuring out what point $M$ on line $L$ will give us the minimum value of $AM+BM$ . Since distances are positive, that minimum value will give us the minimum value of $(AM+BM)^{2}$

Since our points are on the same side of the line, we simply can't connect them together and find our minimum distance, so we need to do something more clever.

If we reflect point $A$ over line $L$ to a point $A'(-3,5)$ , we know that both $A$ and $A'$ are equidistant to any point on line $L$ . We also know that the shortest possible distance between any two points is a straight line, so let's draw a line $P$ that connects $A'$ and $B$ . Let the intersection point between lines $L$ and $P$ be point $M$ .

We get that $AM=A'M$ , and $A'M+BM$ is minimized. Therefore, we can conclude that $AM+BM$ is minimized. We know that $A'M+MB=A'B$ , and by distance formula $A'B=\sqrt{(4+3)^{2}+(5-0)^{2}}=\sqrt{74}$ . Since we want that distance squared, our answer is $\boxed{74}$

---

I was curious as to whether or not that 74 at the very end of our expression being the minimum value was a coincidence or not. Any ideas?

The given expression can be rewritten as $2\sqrt{((x-4)^2+y^2)((x-3)^2+(y-7)^2)}+2((x-\frac72)^2+(y-\frac72)^2)+25$
The point $A(x,y)$ lies on the line $3x+y=6$ . We have two points $B(4,0) \text{ } \& \text{ } C(3,7)$ such that their mid point is $D(\frac72,\frac72)$ So the given expression can be written as $2AB\cdot AC + 2(AD^2+BD^2)$
Using Apollonius' theorem, this can be written as $(AB+AC)^2$
To minimize $AB+AC$ , we find the reflection of point $B$ in the given line and then find the distance of the reflected point from $C$ , since $B\text{ } \& \text{ }C$ are on the same side of the given line.
The reflection of $B(4,0)$ in the line $3x+y=6$ is $B'(\frac25, \frac65)$ .
$B'C^2 = \boxed{74}$ which is the required minimum value.