Hermite, Ostrogradsky

As suggested by the title of the problem, the problem can be solved by Ostrogradsky method . According to the method, if $P (x)$ and $Q (x)$ are polynomials with real coefficients and $P (x) / Q (x)$ is a proper fraction, and $Q (x)$ has multiple roots, then

$\int \frac {P(x)}{Q(x)} dx = \frac {P_1(x)}{Q_1(x)} + \int \frac {P_2(x)}{Q_2(x)} dx$

where $Q_1(x)$ is the greatest common divisor of $Q (x)$ and its derivative $Q' (x)$ , while $Q_2(x) = Q (x) / Q_1(x)$ . Undetermined coefficients of he polynomials $P_1(x)$ and $P_2(x)$ , whose degrees are one less than of the polynomials $Q_1(x)$ and $Q_2(x)$ respectively, we calculate by deriving the above integral identity.

In this problem, $P(x) = x^5+2x^4+4x^3-2x^2-x$ ; $Q(x) = (x+1)^2(x^2+1)^2$ , $Q'(x) = 2(x+1)(x^2+1)^2 + 4x(x+1)^2(x^2+1)$ , then $\gcd (Q(x),Q'(x)) = Q_1(x) = (x+1)(x^2+1)$ and $Q_2(x) = Q(x)/Q_1(x) =(x+1)(x^2+1)=Q_1(x)$ . Now, let $P_1(x) = Ax^2+Bx+C$ and $P_2(x) =Dx^2+ Ex+F$ . Then, we have:

$\begin{aligned} \int \frac {x^5+2x^4+4x^3-2x^2-x}{(x+1)^2(x^2+1)^2} dx & = \frac {Ax^2+Bx+C}{(x+1)(x^2+1)} + \int \frac {Dx^2+ Ex+F}{(x+1)(x^2+1)} dx \quad \quad \small \color{#3D99F6}{\text{Differentiate up and down wrt }x} \\ \frac {x^5+2x^4+4x^3-2x^2-x}{(x+1)^2(x^2+1)^2} & = \frac {(2Ax+B)(x+1)(x^2+1)-(Ax^2+Bx+C)(3x^2+2x+1)}{(x+1)^2(x^2+1)^2} + \frac {Dx^2+ Ex+F}{(x+1)(x^2+1)} \\ x^5+2x^4+4x^3-2x^2-x & = (Dx^2+ (2A+E)x+B+F)(x+1)(x^2+1)-(Ax^2+Bx+C)(3x^2+2x+1) \end{aligned}$

Solving for $A$ , $B$ , $C$ , $D$ , $E$ and $F$ (see below), we get $A = 0, \ B = 0, \ C = 2, \ D = 1, \ E =1, \ F = 2$ . Therefore,

$\begin{aligned} I & = \int_0^2 \frac {x^5+2x^4+4x^3-2x^2-x}{(x+1)^2(x^2+1)^2} dx \\ & = \frac 2{(x+1)(x^2+1)}\bigg|_0^2 + \color{#3D99F6}{\int_0^2 \frac {x^2+x+2}{(x+1)(x^2+1)} dx} & \small \color{#3D99F6}{\text{By partial fractions}} \\ & = - \frac {28}{15} + \color{#3D99F6}{\int_0^2 \left( \frac 1{x+1} + \frac 1{x^2+1} \right) dx} \\ & = - \frac {28}{15} + \bigg[ \ln (x+1) + \tan^{-1} x \bigg]_0^2 \\ & = - \frac {28}{15} + \ln 3 + \tan^{-1} 2 \end{aligned}$

$\implies A+B+C+D = 28+15+3+2 = \boxed{48}$

---

Solving for $A$ , $B$ , $C$ , $D$ , $E$ and $F$

Equating the coefficient of $x^5$ , $\implies D = 1 \ \ ...(1)$ .

Let $x=-1$ $\implies -1+2-4-2+1 = 0 - (A-B+C)(3-2+1) \implies A-B+C = 2 \ \ ...(2)$ .

Let $x=0$ $\implies 0 = B+F-C \implies B-C+F = 0 \ \ ...(3)$ .

Let $x=i$

$\begin{aligned} \implies i+2-4i+2-i & = 0 - (-A+Bi+C)(-3+2i+1) \\ 4-4i & = (-A+Bi+C)(2-2i) \\ 2 & = -A+Bi+C \\ \implies B & = 0 \ \ ...(4) \\ -A+C & = 2 \ \ ...(5) \end{aligned}$

$\begin{aligned} (2)+(5): \ \ -B+2C = -0_{\color{#3D99F6}{(4)}} + 2C = 4 \implies C = 2 \ \ ...(6) \end{aligned}$

$\begin{aligned} (5): \ \ -A+C = -A+2_{\color{#3D99F6}{(6)}} = 2 \implies A = 0 \ \ ...(7) \end{aligned}$

$\begin{aligned} (3): \ \ B-C+F = 0_{\color{#3D99F6}{(4)}} -2_{\color{#3D99F6}{(6)}} +F = 0 \implies F = 2 \ \ ...(8) \end{aligned}$

Let $x=1$

$\begin{aligned} \implies 1+2+4-2-1 & = (1+E+2)(2)(2) -2(6) \\ 4 & = 4E + 12 - 12 \\ \implies E & = 1 \end{aligned}$

$\displaystyle \int _{ 0 }^{ 2 }{ \frac { { x }^{ 5 }+2{ x }^{ 4 }+4{ x }^{ 3 }-2{ x }^{ 2 }-x }{ { (1+x) }^{ 2 }{ (1+{ x }^{ 2 }) }^{ 2 } } dx } =\int _{ 0 }^{ 2 }{ \frac { { (x-1)x(x+1)(x }^{ 2 }+1)+2{ x }^{ 2 }{ (x+1) }^{ 2 }-4{ x }^{ 2 } }{ { (1+x) }^{ 2 }{ (1+{ x }^{ 2 }) }^{ 2 } } dx }$

$\displaystyle \int _{ 0 }^{ 2 }{ \frac { { (x-1)x(x+1)(x }^{ 2 }+1)+2{ x }^{ 2 }{ (x+1) }^{ 2 }-4{ x }^{ 2 } }{ { (1+x) }^{ 2 }{ (1+{ x }^{ 2 }) }^{ 2 } } dx } =\int _{ 0 }^{ 2 }{ \left( \frac { x(x-1) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -\frac { 4{ x }^{ 2 } }{ { (1+x) }^{ 2 }{ (1+{ x }^{ 2 }) }^{ 2 } } \right) dx }$

$\displaystyle \int _{ 0 }^{ 2 }{ \left( \frac { x(x-1) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -\frac { 4{ x }^{ 2 } }{ { (1+x) }^{ 2 }{ (1+{ x }^{ 2 }) }^{ 2 } } \right) dx } =\int _{ 0 }^{ 2 }{ \left( \frac { { x }^{ 2 }+1-(1+x) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { 2x } }{ { (1+x) }{ (1+{ x }^{ 2 }) } } \right) }^{ 2 } \right) dx }$

$\displaystyle \int _{ 0 }^{ 2 }{ \left( \frac { { x }^{ 2 }+1-(1+x) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { 2x } }{ { (1+x) }{ (1+{ x }^{ 2 }) } } \right) }^{ 2 } \right) dx } =\int _{ 0 }^{ 2 }{ \left( \frac { 1 }{ (x+1) } -\frac { 1 }{ { (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { x+1 } }{ { (1+{ x }^{ 2 }) } } -\frac { { 1 } }{ { (1+x) } } \right) }^{ 2 } \right) dx }$

$\displaystyle \int _{ 0 }^{ 2 }{ \left( \frac { { x }^{ 2 }+1-(1+x) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { 2x } }{ { (1+x) }{ (1+{ x }^{ 2 }) } } \right) }^{ 2 } \right) dx } =\ln { (1+x) } -\arctan { x } +\int _{ 0 }^{ 2 }{ \left( \frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { x+1 } }{ { (1+{ x }^{ 2 }) } } -\frac { { 1 } }{ { (1+x) } } \right) }^{ 2 } \right) dx }$

$\displaystyle \int _{ 0 }^{ 2 }{ \left( \frac { { x }^{ 2 }+1-(1+x) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { 2x } }{ { (1+x) }{ (1+{ x }^{ 2 }) } } \right) }^{ 2 } \right) dx } =\ln { (1+x) } -\arctan { x } -\frac { x }{ 1+{ x }^{ 2 } } +\int _{ 0 }^{ 2 }{ \left( \frac { 1 }{ { (1+{ x }^{ 2 }) } } \right) dx } -\int _{ 0 }^{ 2 }{ { \left( \frac { { { x }^{ 2 }+1+2x } }{ { { (1+{ x }^{ 2 }) }^{ 2 } } } -\frac { { 2 } }{ { (1+{ x }^{ 2 }) } } +\frac { 1 }{ { (1+x) }^{ 2 } } \right) }dx }$

$\displaystyle \int _{ 0 }^{ 2 }{ \left( \frac { { x }^{ 2 }+1-(1+x) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { 2x } }{ { (1+x) }{ (1+{ x }^{ 2 }) } } \right) }^{ 2 } \right) dx } =\ln { (1+x) } -\arctan { x } -\frac { x }{ 1+{ x }^{ 2 } } +\arctan { x } -\int _{ 0 }^{ 2 }{ { \left( \frac { { 2x } }{ { { (1+{ x }^{ 2 }) }^{ 2 } } } -\frac { { 1 } }{ { (1+{ x }^{ 2 }) } } +\frac { 1 }{ { (1+x) }^{ 2 } } \right) }dx }$

$\displaystyle \int _{ 0 }^{ 2 }{ \left( \frac { { x }^{ 2 }+1-(1+x) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { 2x } }{ { (1+x) }{ (1+{ x }^{ 2 }) } } \right) }^{ 2 } \right) dx }$ $\displaystyle \ln { (1+x) } -\arctan { x } -\frac { x }{ 1+{ x }^{ 2 } } +\arctan { x } +\frac { 1 }{ 1+{ x }^{ 2 } } +\arctan { x } +\frac { 1 }{ 1+x }$

$\displaystyle \int _{ 0 }^{ 2 }{ \left( \frac { { x }^{ 2 }+1-(1+x) }{ (x+1){ (1+{ x }^{ 2 }) } } +\frac { 2{ x }^{ 2 } }{ { (1+{ x }^{ 2 }) }^{ 2 } } -{ \left( \frac { { 2x } }{ { (1+x) }{ (1+{ x }^{ 2 }) } } \right) }^{ 2 } \right) dx } =\ln { 3 } -\frac { 28 }{ 15 } +\arctan { 2 }$

$\displaystyle A=28, B=15, C=3$ and $D=2$

$\displaystyle A+B+C+D=28+15+3+2=\boxed{48}$