Conservation

Let the velocities after collision of particles of masses $M$ and $m$ be $v_1$ and $v_2$ respectively. Since the collision is elastic, the momentum and energy before and after collision are conserved.

By the conservation of momentum :

$\begin{aligned} Mv_1 + mv_2 & = Mv_M + mv_m \\ 3v_1 + v_2 & = 3(5) - 2 = 13 \\ \implies v_2 & = 13-3v_1 \end{aligned}$

By the conservation of energy :

$\begin{aligned} \frac 12 Mv_1^2 + \frac 12 mv_2^2 & = \frac 12 Mv_M^2 + \frac 12 mv_m^2 \\ \frac 32 v_1^2 + \frac 12 v_2^2 & = \frac 32(25) + \frac 12 (4) = \frac {79}2 \\ 3 v_1^2 + v_2^2 & = 79 & \small \color{#3D99F6} \text{Putting }v_2 = 13-3v_1 \\ 3 v_1^2 + (13-3v_1)^2 & = 79 \\ 12v_1^2 - 78 v_1 + 90 & = 0 \\ 2v_1^2 - 13 v_1 + 15 & = 0 \\ (2v_1-3)(v_1-5) & = 0 \\ \implies v_1 & = \frac 32 & \small \color{#3D99F6} \text{Since }5 = v_M \\ \implies v_2 & = 13 - 3v_1 \\ & = \boxed{8.5} \text{ m/s} \end{aligned}$

The velocities after the collision of a particle $M$ moving before the collision with velocity $v_1$ and a particle $m$ moving before the collision with velocity $v_2$ are given by:

$u_1 = \dfrac{M-m}{M+m} v_1 + \dfrac{2 m}{M+m} v_2$

$u_2 = \dfrac{2M}{M+m} v_1 - \dfrac{M-m}{M+m} v_2$

using the second of these equations yields $u_2 = 8.5 \text{ m/s}$

I have the most intuitive solution about this one.

Take the system into the center of mass reference frame, which we can do since physic is the same in all inertial frames.

This means finding the total momentum in this frame, and finding a frame where the total moment is zero. (I'll dispense with the units for brevity.)

So $Mv_M+mv_m = 13$ , and $M+m = 4$ .

Go into this CM frame, and we need to lose 13/4 on each object, so the velocities are $v_M = 7/4$ and $v_m=-21/4$ .

In an elastic collision where total momentum is zero the collision outcome is an exchange of momenta, so that means the particles collide and reverse directions:

After collision $v_M = -7/4$ and $v_m = +21/4$ .

Go back to the original frame, and we add back the CM motion:

$v_m = +21/4 + 13/4 = 34/4 = 8.5$