Mr Wilson to the rescue!

$(2p-1)!+2(p!)+p=((2p-1)(2p-2)...(p+2)(p+1)(p-1)!+2(p-1)!+1)p$ where $(2p-1)(2p-2)...(p+2)(p+1)=(p+p-1)(p+p-2)...(p+2)(p+1)=(x+p-1)(x+p-2)...(x+2)(x+1)$ where $x=p$ .

Note that the constant of this polynomial is $(p-1)!$ while its coefficient of $x$ is:

$\dfrac{(p-1)!}{1}+\dfrac{(p-1)!}{2}+...+\dfrac{(p-1)!}{p-2}+\dfrac{(p-1)!}{p-1}=\dfrac{(p-1)!}{1}+\dfrac{(p-1)!}{p-1}+\dfrac{(p-1)!}{2}+\dfrac{(p-1)!}{p-2}+... =\dfrac{p(p-1)!}{1(p-1)}+\dfrac{p(p-1)!}{2(p-2)}+...$ which is divisible by $p$ . Note that the last term involves $\dfrac{p-1}{2}$ which is why $p$ must be an odd prime so that $(p-1)$ must be even.

Now upon substitution of $x=p$ , the coefficient of $x$ will contribute another factor of $p$ other than the existing $x=p$ . Considering that the rest of the polynomial is also divisible by $p^2$ with all terms of power at least $x^2$ , we see that the expression is divisible by $p^2$ except for the constant term $(p-1)!$ .

$(2p-1)(2p-2)...(p+2)(p+1) \equiv (p-1)!$ (mod $p^2$ ) $(2p-1)(2p-2)...(p+2)(p+1)(p-1)!+2(p-1)!+1 \equiv (p-1)!(p-1)!+2(p-1)!+1 \equiv ((p-1)!+1)^2$ (mod $p^2$ )

Considering that $(p-1)! \equiv -1$ (mod $p$ ), we have $p \mid ((p-1)!+1)$ thus $p^2 \mid ((p-1)!+1)^2$ . This means that $p^2 \mid ((2p-1)(2p-2)...(p+2)(p+1)(p-1)!+2(p-1)!+1)$ . Then $p^{2+1} \mid ((2p-1)(2p-2)...(p+2)(p+1)(p-1)!+2(p-1)!+1)p$ which means that $p^3 \mid ((2p-1)(2p-2)...(p+2)(p+1)(p-1)!+2(p-1)!+1)p$ . Indeed, $p^3 \mid ((2p-1)!+2(p!)+p)$ . Thus the remainder when $(2p-1)!+2(p!)+p$ is divided by $p^3$ is $\boxed{0}$ .