Easier than it looks

The triangle covers exactly 1/8th of the full solid angle (everything for $x\ge 0$ , $y\ge 0$ , $z\ge 0$ , but nothing elsewhere). For the inverse square field we are dealing with here the flux only depends on the solid angle, and its value is $\phi=\Omega F_0$ , where $F_0=1/\pi$ is the field strength at unit distance. In our case $\Omega=4\pi/8=\pi/2$ the flux is $\phi=\Omega F_0=1/2$

Let $W$ be the solid region in the first octant bounded by the triangle $T$ and the portion $S$ of the sphere $x^2+y^2+z^2=1$ in the first octant. Since $\nabla \cdot \vec{F} =0$ , the flux of $\vec{F}$ out of $W$ is 0, by Ostrogradsky's theorem. Since the flux of $\vec{F}$ through the coordinate planes is 0, this implies that $\int_T \vec{F} \cdot d\vec{S}=\int_S \vec{F} \cdot d\vec{S}$ , both oriented upward. But $\int_S \vec{F} \cdot d\vec{S}=\int_S \frac{1}{\pi} dS=\frac{1}{\pi}(\text{area of} S)=\frac{4\pi}{8\pi}=\boxed{0.5}$ .

For $0 < \varepsilon < 1$ , let $V_\varepsilon$ be the volume bounded by the triangle $T$ defined by the three vertices of the question, the planes $x=0$ , $y=0$ and $z=0$ , and the sphere $x^2 + y^2 + z^2 = \varepsilon^2$ , and let $S_\varepsilon$ be the octant of that sphere for $x,y,z \ge 0$ . Note that $\mathbf{F} = -\nabla \phi$ , where $\phi(\mathbf{r}) = \frac{1}{\pi r}$ is harmonic away from the origin. The upward-pointing normal for $T$ is then the outward-pointing normal for $T$ as a part of $V_\varepsilon$ . Equip $S_\varepsilon$ with the radially-outward-pointing normal.

Since $\mathbf{F}$ points radially at all times, $\mathbf{F} \cdot d\mathbf{S} = 0$ on any of the planes $x=0$ , $y=0$ or $z=0$ , and we see that $\iint_T \mathbf{F}\cdot d\mathbf{S} - \iint_{S_\varepsilon} \mathbf{F} \cdot d\mathbf{S} \; = \; \iiint_{V_\varepsilon}\nabla \cdot \mathbf{F} \,dV \; = \; -\iiint_{V_\varepsilon} \nabla^2\phi\,dV \; = \; 0$ and hence (as usual, $\Omega$ represents solid angle) $\iint_T \mathbf{F} \cdot d\mathbf{S} \; = \; \iint_{S_\varepsilon}\mathbf{F} \cdot d\mathbf{S} \; = \; \iint_{S_\varepsilon} \frac{1}{\pi \varepsilon^2}dS \; = \; \frac{1}{\pi}\iint_{S_\varepsilon}d\Omega \; = \; \boxed{\tfrac12}$