Inequality and integers

If $|a| \leq 1$ then for any $x < 0$ :

${(x-b)}^{2} = {x}^{2} - 2 b x + {b}^{2} > {x}^{2} > {(ax)}^{2}$

Which means infinite integer solutions for $x$ . So from now on, $|a| > 1$ .

Let's find roots of the equation: ${(x-b)}^{2} = {(ax)}^{2}$

${x}^{2} - 2bx + {b}^{2} = {a}^{2} {x}^{2}$

$(1 - {a}^{2}) {x}^{2} - 2bx + {b}^{2} = 0$

$x = \frac{b \pm \sqrt{{b}^{2} - {b}^{2}(1-{a}^{2})}}{1-{a}^{2}}$

$x = \frac{b \pm \sqrt{{a}^{2} {b}^{2}}}{1-{a}^{2}}$

$x = \frac{b \pm |a| b}{1-{|a|}^{2}}$ (since $b > 0$ )

$x = \frac{b(1 \pm |a|)}{(1+|a|)(1-|a|)}$

$x = \frac{b}{1 \mp |a|}$

Because $|a| > 1$ , $\frac{b}{1 - |a|} < 0 < \frac{b}{1 + |a|}$ . Since $x=0$ is a solution for the inequality, all solutions lie between the roots.

From the condition $\frac{b}{1 + |a|} < \frac{1+a}{1+|a|} < 1$ , which means that $x=1$ cannot be a solution. So, the three integer solutions are $0, -1, -2$ .

That can be if two inequalities are true:

$\begin{cases} {(-2-b)}^{2} > {-2a}^{2} & \mbox{for } x = -2 \\ {(-3-b)}^{2} < {-3a}^{2} & \mbox{for } x = -3 \end{cases}$

$\begin{cases} {(b + 2)}^{2} > {2}^{2}{a}^{2} \\ {(b + 3)}^{2} < {3}^{2} {a}^{2} \end{cases}$

${\frac{b}{2} + 1}^{2} > {a}^{2} > {\frac{b}{3} + 1}^{2}$

$\frac{b}{2} + 1 > |a| > \frac{b}{3} + 1$

Finally we get:

$\frac{a+1}{2} + 1 > \frac{b}{2} + 1 > |a| > \frac{b}{3} + 1$

We are not that interested in $b$ :

$\frac{a+3}{2} > |a|$

The solution for the modal inequality is $a \in (-1, 3)$ . But since $|a| > 1$ , all that is left is $\boxed{a \in (1, 3)}$

All is left is to prove that we can always find a proper value for $b$ :

$\frac{a+1}{2} + 1 > \frac{b}{2} + 1 > |a| > \frac{b}{3} + 1 > 1$

$a + 1 > b > 2|a| - 2 > \frac{2}{3} b > 0$

We just need to take value slightly more than $2|a| - 2$ which is always positive, so that the conditions are met