A simple guess-and-check problem

a^2+b^2 = 29 …(1)

a-b = 3

(a-b)^2 = 9 …(2)

a^2–2ab+b^2=9, or

a^2+b^2 -2ab = 9

29–2ab = 9

2ab = 29–9 = 20

so ab = 20/2 = 10.

$a - b = 3$

$a^2 + b^2 = 29$

$a = 3 + b$

$(3 + b)^2 + b^2 = 29 = (3 + b)(3 + b) + b^2 = 29 = 9 + 3b + 3b + b^2 + b^2 = 29 = 9 + 6b + b^2 + b^2 = 29 = 9 + 6b + 2b^2 = 29 = 2b^2 + 6b - 20 = 0 = (2b + 10)(b - 2) = 0, b = -5, b = 2$

If $b = 2$ , $a^2 + 2^2 = 29 = a^2 + 4 = 29 = a^2 = 29 - 4 = 25, a = \surd 25 = 5$ (or $- 5$ )

If $b = -5$ , $a^2 + (-5)^2 = 29 = a^2 + 25 = 29 = a^2 = 29 - 25 = 4, a = \surd 4 = -2$ (or $2$ )

$ab_1 = 2 * 5 = 10$

$ab_2 = -2 * -5 = 5 * 2 = 10$

$ab_1 = ab_2$

Therefore $\fbox{ab = 10}$

Using the identity: $(a-b)^2+2ab=a^2+b^2$ $\Rightarrow (3)^2+2ab=29 \Rightarrow 9+2ab=29 \Rightarrow 2ab=20 \Rightarrow \boxed{ab=10}$

$\begin{cases} a-b = 3 &\ldots (1) \\ \red{a^2 + b^2 = 29} &\red{\ldots (2)} \end{cases}$

$\begin{aligned} (a-b)^2 &= a^2 + b^2 - 2ab &\blue{\text{[Square (1)]}} \\ 9 &= (\red{29})- 2ab &\blue{\text{[Substitute (2)]}} \\ \boxed{10} &= ab \end{aligned}$