A troublesome point!

Let the vertices be $A(0,3)$ , $B(0,0)$ and $C(4,0)$ and $M(x,y)$ . Then we have:

$\begin{aligned} S & = AM^2+BM^2+CM^2 \\ & = x^2 + (3-y)^2 + x^2+y^2+(4-x)^2+y^2 \\ & = x^2+9-6y+y^2+x^2+y^2+16-8x+x^2+y^2 \\ & = 3x^2+3y^2-8x-6y+25 \\ & = 3 \left( x^2 - \frac 83 x + \frac {16}9 \right) + 3 \left(y^2 -2y+1 \right) + \frac {50}3 \\ & = 3 \left( x - \frac 43 \right)^2 + 3 \left(y -1 \right)^2 + \frac {50}3 \end{aligned}$

$\implies S$ is minimum, when $x - \dfrac 43 = 0$ and $y-1=0$ and $S_{min} = \dfrac {50}3$ . $\implies a+b = 50+3 = \boxed{53}$

Let S = AM² + BM² + CM², S = 3x²+3y²-8x-6y+25, S is a partial funtion of x and y , to minimize this function ; First , take partial derivatives with respect to x and equate it to zero : ds/dx = 6x-6 =0 , x =1 the second step is to take partial derivatives with respect to y and equate it to zero : ds/dy = 6y-8 =0 , y = 8/6, substitute the values of x and y int the main function of S , hence S= 50/3 = a/b , a+b = 53

Put masses of $1 kg$ each on the three vertices $A,B,C$ . The centre of mass of this system is simply $\frac{A+B+C}{3}$ , which is the centroid $G$ of the triangle.

Take any point $M$ in the plane of the triangle, make an axis $a_{M}$ through it and orthogonal to the plane of the triangle. $a_{G}$ is defined similarly.

By parallel-axes theorem,

$I_{M} = I_{G} + 3(MG)^{2}$ , or

$AM^{2} + BM^{2} + CM^{2} = AG^{2} + BG^{2} + CG^{2} + 3(MG)^{2}$ , so the given expression takes the least value when $MG=0$ , or when $M$ is the centre of mass $G$ itself. This value is given by $AG^{2} + BG^{2} + CG^{2} = \frac{AB^{2} + BC^{2} + CA^{2}}{3} = \frac{50}{3}$ .

Therefore $a+b = 50+3 = 53$ .