$a^{2}+b^{2}$

Without loss of generality, $a,b > 0$ . We can write $a\sin x + b\cos x \; = \; R\sin(x + \epsilon) \hspace{2cm} a\cos x - b\sin x \; = \; R\cos(x + \epsilon)$ where $R = \sqrt{a^2 + b^2}$ and $0 < \epsilon < \tfrac12\pi$ and $\tan\epsilon = \tfrac{b}{a}$ . Thus we want to maximize $|R\sin(x+\epsilon)-1| + |R\cos(x+\epsilon)| \hspace{2cm} x \in \mathbb{R}$ which is the same as maximizing $f(x) \; = \; |R\sin x - 1| + R|\cos x| \hspace{2cm} x \in \mathbb{R}$ We note that $0 \le f(x) \le R(|\sin x| + |\cos x|) + 1 \; \le \; R\sqrt{2} + 1 \hspace{2cm} x \in \mathbb{R}$ and we note that $f(-\tfrac14\pi) \; = \; R\sqrt{2} + 1$ and so the maximum value of $f(x)$ is $R\sqrt{2} + 1$ . Since this is equal to $11$ , we deduce that $R = 5\sqrt{2}$ , so $a^2 + b^2 = \boxed{50}$ .