Know your roots

Assuming that the roots are imaginary, $\Delta < 0$

$\displaystyle 4\cot^2B - 4 < 0 \Rightarrow \cos^2B<\sin^2B \Rightarrow \sin^2B>\frac{1}{2}$

It is given that $\sin A$ , $\sin B$ and $\cos A$ are in a geometric progression, thus $\displaystyle \frac{\cos A}{\sin B} = \frac{\sin B}{\sin A}$

Sustituding $\sin^2B = \sin A \cos A$ , we have $\displaystyle \sin A \cos A>\frac{1}{2}\Rightarrow\sin2A>1$ , which leads to a contradiction.