How many license tags?

First, let‘s make a list of all possible systems.

If there are $a$ letters and $b$ digits, the system will give $26^a \cdot 10^b$ possibilities.

So, for $a=0$ letters, $b$ must satisfy $26^0 \cdot 10^b \geq 10,000$ . Because of rule 3, we have to pick the smallest $b$ , which is $b=4$ . This means that the system $S(0,4)$ is allowed. We do the same thing for $b=1,2,…$ to get the valid systems $S(1,3)$ , $S(2,2)$ and $S(3,0)$ .

Here we can stop because $a \geq 4$ would break rule 3. By rule 5, $S(3,0)$ is also invalid because it doesn‘t use any digits.

Therefore, we are left with $S(0,4)$ , $S(1,3)$ and $S(2,2)$ .

• $S(0,4)$ : There are no letters so rule 4 doesn‘t restrict anything. $10^4$ is a perfect square, but it uses 5 digits, so there are $\sqrt{10^4} - 1= 100$ squares remaining. But we shouldn‘t forget about 0, so actually there are 100 perfect squares which means 100 possibilities.
• $S(1,3)$ : There are 26 possibilities for the single letter. Similar to $10^4$ , for $10^3$ we also can take the squareroot and round up to get $\bigl\lceil \sqrt{10^3} \bigl\rceil = 32$ perfect squares. In total, this system has $26 \cdot 32 = 832$ tags to reserve.
• $S(2,2)$ : As before, we have 26 possible letters (by rule 4 both letters are the same) and $\bigl\lceil \sqrt{10^2} \bigl\rceil = 10$ perfect squares, so there are $26 \cdot 10 = 260$ license tags.

So in total, from all valid systems they can reserve $100 + 832 + 260 = \boxed{1192}$ .

(Note: $\lceil x \rceil$ means rounding up to the next integer)

I treated "rule 5 slightly differently: I treated it as prohibiting company reserved tags. That does not change the answer. If the same rules are used, then five systems pass: {0,6,1000000,1000},{1,5,2600000,8242},{2,4,6760000,2600},{3,2,1757600,260},{4,1,4569760,104} and the number of company reserved tags is 12206.