Inscribed Square in Triangle

Referring to the picture, it should be clear that we have some similar triangles above and to the right of the square. Letting the side length of the square be $x$ , we can write

$\frac{4-x}{x}=\frac{x}{3-x}.$

Expanding and solving gives us $x=\frac{3 \times 4}{3+4}=\frac{12}{7}$ . Thus the answer is $x^2=144/49 \rightarrow 144+49=\boxed{193}$ .

But if you inscribe about the hypotenuse the area come out to be 4

There are two ways to inscribe a square in the right triangle: based along the hypotenuse or based along the legs. The largest square is based along the legs.
Let $x$ be the side length of the incribed square. We have $\dfrac{4-x} { x} = \dfrac{4}{3}$ , therefore, $x= \dfrac{12}{7},$ and $x^2 = \dfrac{144}{49}.$ Thus $a+b= 144+49=\boxed{193}$ .