Challenging Potential AIME Problem

First, I will attempt to solve the arbitrary case of $c$ being the longest leg and $a$ and $b$ being the shorter legs. By the Pythagorean Inequalities, $a^{2}+b^{2}<c^{2}$ . From AM-GM, $\frac {a^{2}+b^{2}}{2} \geq \sqrt{a^{2}b^{2}}= ab$ . So, $a^{2}+b^{2} \geq 2ab$ . Therefore, $2ab < c^{2}$ and $ab < \frac {c^{2}}{2}$ . So in this case, $c = 18$ so $ab < \frac {18^{2}}{2} = 162$ . The triangle also has to satisfy the inequality $a+b>c$ , so $a+b>18$ . The smallest this could happen at is $a = 0.00000...1$ and $b = 18$ (hypothetically). So the smallest product is 0. In interval notation, this is $\left(0, 162\right)$ , so $b = \boxed {162}$ .

Let the sides be $p,q$ and $r$ and the largest angle be $\theta$ . Since $\theta$ is obtuse, $\cos \theta < 0$ . $r=18$

Hence, $\dfrac{p^2+q^2-r^2}{2pq} <0$ $\Rightarrow p^2+q^2 <r^2$

Also, $p^2 + q^2 \geq 2pq$ . Hence,

$pq < \dfrac{r^2}{2} = 162$

Make the longest side of the triangle the diameter of a circle. Any right triangle whose hypotenuse shares this diameter and whose remaining apex is on the circle bounds the obtuse triangles within it. Thus the set of right triangles bounds the set of obtuse triangles. The maximal right triangle is the isosceles triangle. It's sides are 9 and 9 and they contain a right angle. And so the result.

I solved it using the fact that the bounds of an obtuse triangle is the degenerate triangle and the right triangle, that is, having an angle of 180 and 90. The product of legs of a degenerate triangle is obviously 0, and the product of legs of a right triangle with hypotenuse 18 is $(9\sqrt{2})^2$ , or $162$ .