Similar matrices

One way to solve this problem is to use Jordan canonical form . Two matrices are similar if and only if they have the same Jordan canonical form (up to rearrangement of the Jordan blocks).

To compute the Jordan canonical form, the idea is to look at the dimensions of the kernels $N(A-\lambda I)^m$ where $\lambda$ is an eigenvalue of $A.$ The only eigenvalue of each of these matrices is $1,$ since the characteristic polynomial is $(t-1)^3.$

The kernel $N(A-I)$ is one-dimensional for $A = \begin{pmatrix} 1&1&1 \\ 0&1&1 \\ 0&0&1 \end{pmatrix},$ but for the other three matrices, the kernel $N(A-I)$ is two-dimensional: e.g. for $A = \begin{pmatrix} 1&0&1 \\ 0&1&1 \\ 0&0&1 \end{pmatrix}$ , the kernel of $A-I$ is spanned by $\begin{pmatrix} 1\\0\\0 \end{pmatrix}, \begin{pmatrix} 0\\1\\0 \end{pmatrix}.$ In other words, the $1$ -eigenspace of $\begin{pmatrix} 1&1&1 \\ 0&1&1 \\ 0&0&1 \end{pmatrix}$ is not the same dimension as the $1$ -eigenspaces of the other three.

This is enough to choose the right answer already, since the dimension of an eigenspace is invariant under similarity. Let's just complete the answer by showing that the other three matrices are in fact similar to each other.

This involves also looking at the dimension of $N((A-I)^2).$ For $A = \begin{pmatrix} 1&1&1 \\ 0&1&1 \\ 0&0&1 \end{pmatrix},$ $(A-I)^2 = \begin{pmatrix} 0&0&1 \\ 0&0&0 \\ 0&0&0 \end{pmatrix},$ so $N((A-I)^2)$ is two-dimensional, but for the other three matrices, $(A-I)^2$ is the zero matrix, so its kernel is three-dimensional.

Finally, $(A-I)^3$ is the zero matrix for all four possibilities, so the kernel is three-dimensional in all cases.

So the sequence $w_{1,1},w_{1,2},w_{1,3},\ldots$ of dimensions of the respective kernels is $1,2,3,3,\ldots$ for $\begin{pmatrix} 1&1&1 \\ 0&1&1 \\ 0&0&1 \end{pmatrix},$ but for the other three matrices it is $2,3,3,3,\ldots.$

So the Jordan canonical form of the odd matrix out consists of one block $\begin{pmatrix} 1&1&0 \\ 0&1&1 \\ 0&0&1 \end{pmatrix},$ but the Jordan canonical form of the other three consists of two blocks, one of size 2 and one of size 1: $\begin{pmatrix} 1&1&0 \\ 0&1&0 \\ 0&0&1 \end{pmatrix}.$ Since the other three all have the same Jordan canonical form, which they are all similar to, they are similar to each other as well.