Primes and Square Roots do not mix

Firstly, if $b$ is odd, then $2a-b$ is odd, so $\frac{b}{4} \sqrt{\frac{2a-b}{2a+b}}$ results in an irreducible fraction with a denominator divisible by 4. Thus, $b$ is even. We will now consider 2 cases: when $b=4k$ for some positive integer $k$ , or when $b=2m$ , where $m$ is an odd positive integer.

Case 1: $b=4k$

Substituting this in, we get

$\begin{aligned} p = \dfrac{b}{4} \sqrt{\dfrac {2a-b}{2a+b}} &= \dfrac {4k}{4} \sqrt{\dfrac{2a-4k}{2a+4k}}\\ &= k \sqrt{\dfrac{a-2k}{a+2k}}\\ \left ( \dfrac{p}{k} \right )^2 &= \dfrac {a-2k}{a+2k}\\ p^2 (a+2k) &= k^2 (a-2k)\\ a (k^2 - p^2) &= 2k (k^2 + p^2)\\ a &= \dfrac{2k (k^2 + p^2)}{k^2-p^2} \end{aligned}$

We know $a$ must be an integer. We will consider 2 sub-cases now: when $p \mid k$ and when $p \not \mid k$ .

Case 1.1: $p \mid k$

Let $k=dp$ . We have

$\begin{aligned} a = \dfrac{2k (k^2 + p^2)}{k^2-p^2} &= \dfrac {2dp (d^2 p^2 + p^2)}{d^2 p^2 - p^2}\\ &= \dfrac {2dp (d^2 + 1)}{d^2 - 1} \end{aligned}$

We have $\gcd (d, d^2 - 1)=1$ , so

$\begin{aligned} \dfrac{2p(d^2+1)}{d^2-1} = 2p \left ( 2 + \dfrac {1}{d^2 - 1} \right ) &\in \mathbb{Z}\\ \implies \dfrac {4p}{d^2 - 1} &\in \mathbb{Z} \end{aligned}$

If $d$ is even, we have $d^2-1$ is odd, so $d^2 - 1 \mid p \implies p = d^2 - 1 = (d+1)(d-1)$ . This is only possible if $d-1=1$ , so $d=2, p=3$ . Substituting this into the original formulae, we get $(a,b,p)=(20,24,3)$ .

When $d$ is odd, we must have $d^2 - 1\mid 4p \implies 4p = d^2 - 1 \implies d=3, p=2$ . Therefore, we get $(a,b,p) = (15,24,2)$ .

Case 1.2: $p \not \mid k$

$\gcd (k, k^2 - p^2) = \gcd(k, p^2) = 1$ . Therefore,

$\begin{aligned} \gcd (2k (p^2 + k^2), k^2 - p^2) &= \gcd(2p^2 + 2k^2, k^2 - p^2)\\ &= \gcd (4p^2, k^2 - p^2) \end{aligned}$

Furthermore, $\gcd(p^2, k^2 - p^2) = \gcd(p^2, k^2) = 1$ . Thus,

$\gcd (4p^2, k^2 - p^2) = \gcd(4, k^2 - p^2)$

Thus, we want $k^2 - p^2 = 1, 2, 4$ . However, none of these values gives a positive solution for $(k,p)$ , so there are no solutions in this case.

Case 2: $b=2m$

$\begin{aligned} p = \dfrac {b}{4} \sqrt{\dfrac {2a-b}{2a+b}} &= \dfrac {2m}{4} \sqrt {\dfrac {2a - 2m}{2a + 2m}}\\ &= \dfrac {m}{2} \sqrt{ \dfrac {a-m}{a+m}}\\ \left ( \dfrac{2p}{m} \right ) ^2 &= \dfrac {a-m}{a+m}\\ 4p^2 (a+m) &= m^2 (a-m)\\ a (m^2 - 4p^2) &= m (m^2 + 4p^2)\\ a &= \dfrac {m(m^2+4p^2)}{m^2-4p^2} \end{aligned}$

We do a similar case breakup as previously:

Case 2.1: $p \mid m$

Let $m=dp$ . We have

$\begin{aligned} a = \dfrac {m (m^2 + 4p^2)}{m^2 - 4p^2} &= \dfrac{dp (d^2 p^2 + 4p^2)}{d^2 p^2 - 4p^2}\\ &= \dfrac {dp(d^2 + 4)}{d^2 - 4} \end{aligned}$

Note that since $m$ is odd, $d$ must also be odd. Therefore, $\gcd(d, d^2 - 4) = \gcd(d, 4) = 1$ and $\gcd(d^2 - 4, d^2 + 4) = \gcd(d, 8) = 1$ . herefore, in order for $a$ to be an integer, $\frac {p}{d^2 - 4}$ must be an integer. Thus, $p=d^2 - 4 \implies p = (d-2)(d+2) \implies d-2=1$ . From this, we get $(p,d) = (5,3)$ , so $(a,b,p)=(39,30,5)$ .

Case 2.2: $p \not \mid m$

We have $\gcd(m, m^2 - 4p^2) = \gcd(m, 4p^2) = 1$ since $m$ is odd. Therefore, for $a$ to be an integer, we must have

$\dfrac{m^2 + 4p^2}{m^2 - 4p^2} = 1 + \dfrac {8p^2}{m^2 - 4p^2} \in \mathbb{Z}$

Furthermore, $\gcd(m^2 - 4p^2, p^2) = 1$ since $m$ is odd, so we must have $m^2 - 4p^2 \mid 8$ . Thus, we must have $m^2 - 4p^2 = 1,2,4,8$ . However, if we check these cases, we find no valid integer solutions.

---

Therefore, over all the cases, we have 3 solutions, namely $(20, 24, 3), (15, 24, 2), (39, 30, 5)$ .