an obscure triangle

The nine-point circle passes through the foot of each altitude and through the midpoint of the line segment from each vertex of the triangle to the orthocentre, thus, in order for the circle to pass through the orthocentre as well, one of these segments must be of zero length. Otherwise, on every altitude, these three points would be collinear and the nine-point circle couldn’t pass through them. This means that the orthocentre will coincide with one vertex of the triangle, i.e. the triangle has to be a right-angled triangle. The smallest integer right triangle is the $3$ - $4$ - $5$ triangle, thus the answer is $3+4+5=\boxed{12}$ .

Recall that the nine point centre $N$ is the midpoint of $OH$ , where $O$ is the circumcentre and $H$ is the orthocentre; also the radius of the nine point circle is $\tfrac12R$ , where $R$ is the circumradius.

Then $H$ will lie on the nine point circle precisely when $NH = \tfrac12R$ , namely when $OH = R$ , so that $H$ lies on the circumcircle. But $OH^2 \; = \; R^2\big(1 - 8\cos A \cos B \cos C\big)$ and hence we deduce that $\cos A \cos B \cos C = 0$ , so that the triangle is right-angled. The smallest integer-sided right-angled triangle has sides $3,4,5$ , making the answer $\boxed{12}$ .

---

Here is a proof of the formula for $OH$ . Note that $\begin{aligned} \sin^2A + \sin^2B + \sin^2C & = \; \sin^2A + \sin^2B + (\sin A \cos B + \cos A \sin B)^2 \\ & = \; \sin^2A + \sin^2B + (1 - \cos^2A)\cos^2B + \cos^2A(1 - \cos^2B) + 2\sin A \cos A \sin B \cos B \\ & = \; 2 - 2\cos^2A\cos^2B + 2\sin A \cos A \sin B \cos B \; = \; 2 - 2\cos A \cos B(\cos A \cos B - \sin A \sin B) \\ & = \; 2 + 2\cos A \cos B \cos C \end{aligned}$ If we set up our origin at the circumcentre $O$ , and let $A,B,C$ have position vectors $\mathbf{a},\mathbf{b},\mathbf{c}$ , then $H$ has position vector $\mathbf{a}+\mathbf{b}+\mathbf{c}$ , and hence $\begin{aligned} OH^2 & = \; \big|\mathbf{a}+\mathbf{b}+\mathbf{c}\big|^2 \; = \; 3R^2 + 2(\mathbf{a}\cdot\mathbf{b} + \mathbf{a}\cdot\mathbf{c} + \mathbf{b} \cdot \mathbf{c}) \\ & = \; R^2\big(3 + 2\cos2A + 2\cos2B + 2\cos2C\big) \; = \; R^2\big(4(\cos^2A + \cos^2B + \cos^2C) - 3\big) \\ & = \; R^2\big(9 - 4(\sin^2A + \sin^2B + \sin^2C)\big) \; = \; R^2\big(1 - 8\cos A \cos B \cos C\big) \end{aligned}$