JEE advanced type

Let $\cot^{-1} \left(\sqrt{x-1}\right) = \tan^{-1} \left( \dfrac{1}{\sqrt{x-1}} \right) = \theta$ .

$\begin{aligned} \Rightarrow \tan \theta & = \frac{1}{\sqrt{x-1}} \\ \sec^2 \theta \space d\theta & = -\frac{1}{2(\sqrt{x-1})^3} \space dx = -\frac{\tan^3 \theta}{2} \space dx \\ \Rightarrow dx & = -\frac{2 \cos \theta}{\sin^3 \theta} \space d \theta \end{aligned}$

Then, we have:

$\begin{aligned} I & = \int_1^2 \left( \cot^{-1} \left(\sqrt{x-1}\right) \right)^2 dx \\ & = 2 \int_\frac{\pi}{4}^\frac{\pi}{2} \frac{\color{#D61F06}{\theta^2} \color{#3D99F6}{\cos \theta}}{\color{#3D99F6}{\sin^3 \theta}} \color{#3D99F6}{\space d \theta} \quad \quad \small \color{#3D99F6}{\text{By integration by parts: }} \color{#D61F06}{u = \theta^2}, \space \color{#3D99F6}{dv = \frac{\cos \theta}{\sin^3 \theta} \space d \theta } \\ & = 2\left[-\frac{\theta^2}{2\sin^2 \theta} \right]_\frac{\pi}{4}^\frac{\pi}{2} + 2 \int_\frac{\pi}{4}^\frac{\pi}{2} \frac{\color{#D61F06}{\theta}}{\color{#3D99F6}{\sin^2 \theta}} \color{#3D99F6}{\space d \theta} \quad \quad \small \color{#D61F06}{u = \theta}, \space \color{#3D99F6}{dv = \frac{d \theta}{\sin^2 \theta}} \\ & = - \frac{\pi^2}{8} + 2 \left[-\theta \cot \theta \right]_\frac{\pi}{4}^\frac{\pi}{2} + 2 \int_\frac{\pi}{4}^\frac{\pi}{2} \cot \theta \space d \theta \\ & = - \frac{\pi^2}{8} + \frac{\pi}{2} + 2 \left[\ln (\sin \theta)\right]_\frac{\pi}{4}^\frac{\pi}{2} \\ & = - \frac{\pi^2}{8} + \frac{\pi}{2} + \ln 2 \end{aligned}$

Therefore,

$\begin{aligned} \Rightarrow \frac{\pi^2+8I-8\ln 2}{\pi} & = \frac{\pi^2 + 8 \left( - \frac{\pi^2}{8} + \frac{\pi}{2} + \ln 2\right) -8\ln 2}{\pi} \\ & = \boxed{4} \end{aligned}$

I= $\int _{ 1 }^{ 2 }{ ({ arccot ^{ }{ sqrt {( x-1) } ) } }^{ 2 } }$ Put x-1 = ${ t }^{ 2 }$ $I=\int _{ 0 }^{ 1 }{ t\times ({ arccot ^{ }{ t) } }^{ 2 }dt }$ => dx=2tdt
Using integration by parts we can choose t as the first function and arccot(t) as the second function. You will get the final answer as $-\frac { { \pi }^{ 2 } }{ 8 } +\frac { \pi }{ 2 } +\ln { 2 }$ . Then by solving the fraction and rearranging the terms you will get the answer as 4. Please tell me if anyone needs the complete solution.