A small inverse

It might seem reasonable that the inverse of $M$ involve the inverse of $A.$ Let's see.

Suppose the inverse of $M$ is $M^\prime = I - A^{-1}.$ Then, multiplying on the left (or right) we have $M^\prime M = (I + A)(I - A^{-1}) = I - A + A^{-1} + I = 2 - A + A^{-1}$ which is clearly not the identity matrix, $I.$ We'd get a similar outcome if we tried $M^\prime = I + A^{-1}.$

By assumption, the elements of $A^2$ are negligible so that $A^2\approx 0,$ and we can ignore any second order terms in $A.$ Therefore, as long as we can get a cancellation of $A$ to first order, we can construct the desired inverse.

Let's try squaring the original matrix $M.$ $M^2 = (I + A)^2 = I^2 + A + A + O(A^2) = I + 2A + O(A^2)$ That didn't work, but we can see that if the second $A$ were negated, we'd get the cancellation we need. Thus the inverse is given by $M^\prime = I - A.$

$M^\prime M = \left(I - A\right)\left(I + A\right) = I + A - A + O(A^2) = I + O(A^2) \approx I.$