Order them balls

The Johnson-Trotter algorithm provides a Hamiltonian path of the permutohedron of $S_n$ , so that it is possible to obtain all $n!$ elements of $S_n$ by applying $0,1,2,...,n!-1$ transpositions successively. Thus the correct permutation can be achieved in at most $n!-1$ switches. In this case, the answer is $\boxed{719}$ .

The Johnson-Trotter algorithm works by successively placing $n$ in all the possible positions within all the possible permutations of $1,2,...,n-1$ , and building up inductively. Thus we have $\begin{aligned} 1\mathbf{2} \\ \mathbf{2}1 \end{aligned}$ for $n=2$ , then $\begin{aligned} 12\mathbf{3} \\ 1\mathbf{3}2 \\ \mathbf{3}12 \\ \mathbf{3}21 \\ 2\mathbf{3}1 \\ 21\mathbf{3} \end{aligned}$ for $n=3$ , and then $\begin{aligned} 123\mathbf{4} \\ 12\mathbf{4}3 \\ 1\mathbf{4}23 \\ \mathbf{4}123 \\ \mathbf{4}132 \\ 1\mathbf{4}32 \\ 13\mathbf{4}2 \\ 132\mathbf{4} \\ 312\mathbf{4} \\ \vdots \\ 213\mathbf{4} \end{aligned}$ and so on.

There are 6! = 720 ways of sorting the balls, so the best we can achieve is 719 switches, since each switch produces one new possibility.

And this can be achieved recursively for N balls by switching out each of the N colors to be in the first position and sorting the remaining N-1 similarly.