Gram-Schmidt Or?

For all real $x$ , $|x|^2 = x^2$ , so

$\begin{aligned} I_{a,b,c} &= &\int_{-1}^1 |x^3 - a-bx - cx^2|^2 \, dx \\ &=& \int_{-1}^1 (x^3 - a-bx - cx^2)^2 \, dx \\ &=& \int_{-1}^1 [ x^6 + x^5(-2c) + x^4(-2b + c^2) + x^3(-2a +2bc) + x^2(2ac+b^2) + x(2ab) + a^2 ] \, dx \\ &=& \left [\dfrac17x^7 + \dfrac{-2c}6 x^6 + \dfrac{-2b+c^2}5 x^5 + \dfrac{-2a+2bc}4 x^4 + \dfrac{2ac+b^2}3 + \dfrac{2ab}2 x^2 + a^2 x \right]_{x=-1}^{x=1} \\ &=& \dfrac2{105} (105a^2 + 70ac + 35b^2 - 42b + 21c^2 + 15) \end{aligned}$

What's left is to minimize the expression $105a^2 + 70ac + 35b^2 - 42b + 21c^2 + 15$ via completing the square ,

$\begin{aligned} 105a^2 + 70ac + 35b^2 - 42b + 21c^2 + 15 &=& (105a^2 + 70ac + 21c^2) + (35b^2 - 42 b + 15) \\ &=& 105 \left( a^2 + \dfrac23 ac + \dfrac15 c^2 \right) + 35 \left( b^2 - \dfrac65 b + \dfrac37\right) \\ &=& 105 \left [ \left( a + \dfrac c3\right)^2 + \dfrac{c^2}5 - \dfrac{c^2}9 \right] + 35 \left[ \left( b - \dfrac35 \right)^2 - \dfrac{9}{25} + \dfrac37\right] \\ &=& 105 \left [ \left( a + \dfrac c3\right)^2 - \dfrac{4c^2}{45} \right] +35 \left( b - \dfrac35\right) ^2 + \dfrac{12}5 \\ &=& 105 \left( a + \dfrac c3\right)^2 + \dfrac{28}3 c^2 + 35 \left( b - \dfrac35\right) ^2 + \dfrac{12}5 \\ & \geq & 0 + 0 + 0 + \dfrac{12}5 = \dfrac{12}5 \; , \end{aligned}$

because $x^2 \geq 0$ for all real $x$ .

Hence, $I_{a,b,c} \geq \dfrac{2}{105} \cdot \dfrac{12}5 = \dfrac8{175}$ when $a + \dfrac c3 = b - \dfrac 35 = c = 0 \Rightarrow (a,b,c) = \left( 0,\dfrac35,0\right)$ . The answer is $8 + 175 = \boxed{183}$ .

The minimum is got when $a =0, b = 3/5 , c=0$ and $\frac{A}{B} = \frac{8}{175}$ . Basically, the proof is to find the best aproximattion(approach) from $x^3$ to the vector space of the polynomials of second grade with an inner product....

---

Subparagraph before solution

In math we have to fight to avoid ambiguities. (Sorry for my English and some names)

Definition.- A set is a well-defined collection of objects or elements. For more information you can consult, Set, mathematical definition . There was a controversy at the beginning of the 20th century (fundamental crisis), because of the definition of a "set". For more information, you can consult Russell's paradox . I recommend an algebra book to get you started: Algebra (Thomas Hungerford).

Definition.- A nonempty set $A$ with two operations ( $a + b, a \times b$ ) is said to be a conmutative ring if it satisfies:

1.- Properties of $+$ :

$\space \space$ 1.a) Associative property: $a + (b + c) = (a + b) + c, \space \forall a, b , c \in A$ .

$\space \space$ 2.a) Commutative property: $a + b = b + a, \space \forall a, b \in A$ .

$\space \space$ 3.a) $\exists$ an neutral element for $+$ (we'll call it $0$ ), such that $a + 0 = a = 0 + a, \space \forall a \in A$ .

$\space \space$ 4.a) $\forall a \in A \space \exists$ an inverse element for $+$ , i.e, $\forall a \in A \space, \exists \space -a$ , such that, $a + (-a) = 0 = (-a) + a$ .

2.- Properties of $\times$ :

$\space \space$ 1.a) Associative property: $a \times (b \times c) = (a \times b) \times c, \space \forall a, b , c \in A$ .

$\space \space$ 2.a) Commutative property: $a \times b = b \times a, \space \forall a, b \in A$ . Because of this property, the ring is commutative.

3.- Properties with $+$ and $\times$ :

$\space \space$ Distributive properties:

$\forall a, b, c, \in A, \space a \times (b + c) = a \times b + a \times c$ .

$\forall a, b, c, \in A, \space (a + b) \times c = a \times c + b \times c$ .

If there exists an identity element $1$ for $\times$ , i. e, $a \cdot 1 = a = 1 \cdot a, \forall a \in A$ , we'll say that $A$ is a commutative ring with identity .

A element $a^{-1} \in A$ such that $a^{-1} \cdot a = a \cdot a^{-1} = 1$ is an inverse element of $a$ under $\times$ .

A commutative ring with identity in which every nonzero element (each element distinct to $0$ ) has an inverse element is called a field .

Definition.- A nomempty set $E$ is said to be a vector space over a field $K$ ... (Pause, to be continued?)

Definition.- A norm in a vector space $V$ over a field $K$ . (For us $K$ will be $\mathbb{R}$ or $\mathbb{C}$ )is a function $|| \cdot ||: V \to \mathbb{R}^{+}$ such that satisfies:

a) $|| x || \ge 0, \space \forall x \in V$ , and $||x|| = 0 \iff x = 0$

b) $|| x + y || \leq ||x || + || y ||, \space \forall x , y \in V$

c) $||\lambda \cdot x || = |\lambda| \cdot ||x||, \space \forall x \in V$

Definition.- A normed vector space, is a vector space with a norm.

Associated with the norm we will have a distance : $d(x,y) = || x - y ||, \space \forall x, y \in V$ . You can prove :

a) $d(x, y) \ge 0, d(x, y) = 0 \iff x = y$

b) $d(x,y) = d(y,x)$

c) $d(x,z) \leq d(x,y) + d(y,z)$ (triangular inequality) $d(x,z) = || x - z || = || x - y + y - z || \leq ||x - y|| + ||y - z|| = d(x,y) + d(y,z)$ .

Examples.-

a) $A$ = $(\mathbb{R}, | \cdot |)$ is a normed vector space over $\mathbb{R}$ , where $|\cdot |$ is the absolute value. Furthemore, $A$ is a Banach space, this means a normed vector space where all Cauchy sequence converges. The reciprocal is always true,i.e, if a sequence converges(in a normed vector space) it's a Cauchy sequence.

b) $\mathcal{C}([-1,1]) = \{ f: [-1, 1] \to \mathbb{R}; \text{f is continuous}\}$ is a vector space over $\mathbb{R}$ , with the norm $\sqrt{\int_{-1}^1 f(t)^2 \, dt}$ . This example is which we need in this exercise. We shall see that this norm comes from an inner product "dot product" and prove that it is really a norm.

We just only need $\boxed{\text{3 theorems}}$ and the notion of orthoganillity and ortonormallity

Definition.-

Two vectors $v$ and $w$ are said to be orthogonal if $v \cdot w = \langle v , w \rangle = 0$ . They are orthonormal if each vector has length $1$ , i.e., v is orthonormal if $\langle v, v \rangle = 1$ . Notation.- $x \bot y$ means $x$ is orthogonal to $y$ , i. e, $\langle x, y \rangle = 0$

Definition.- $(E, \langle \cdot, \cdot \rangle)$ is a prehilbert space iff $E$ is a vectorial space and $\langle \cdot, \cdot \rangle$ is an inner product.

Now, I'm going to give the definitions and necessary theorems for solving this problem without demostrations, and after I'll give the solution. Finally, I'll give the proofs of these theorems, and maybe I keep on working on this site, creating a wiki...

Theorem 1.- Cauchy- Schwartz inequality Generalisation

Let $(E, \langle \cdot, \cdot \rangle)$ be a prehilbert space. Then:

a) $\forall x, y \in E$ , $|\langle x ,y \rangle|^2 \leq \langle x ,x \rangle \cdot \langle y ,y \rangle$ ( Cauchy-Schwartz inequality)

b) We can define a norm in $E$ . Thus,If $|| x || = + \sqrt{\langle x ,x \rangle}, \space \forall x \in E$ , then $|| \cdot ||$ is a norm.

Definition.- Let $(E, ||\cdot||)$ bea normed vectorial space, $S \subseteq E$ a subset and $x \in E$ , then:

a) The distance from $x$ to $S$ is $d(x,S) = \text{ infimum} \{d(x,y), \space y \in S\}$ = $\text{ infimum } \{ ||x - y||,\space y \in S \}$

b) If there exists $y \in S$ such that $d(x, S) = d(x, y) = || x - y||$ , $y$ is a best approximation(approach) from $x$ to $S$ .

Examples.-

1.- In $\mathbb{R}^2$ , $S = \{ (x, y) \in \mathbb{R}^2, x^2 + y^2 < 1\}$ , and let $x = (0, 2)$ then $d(x, S) = 1$ , nevertheless there doesn't exist a best approximation from $x$ to $S$ .

2.- In $\mathbb{R}^2$ , $S = \{ (x, y) \in \mathbb{R}^2, x = 0 \vee y = 1\}$ , and let $x = (1,0)$ then $d(x, S) = 1$ and there exist 2 best approximation from $x$ to $S$ : $y_1 = (0,0) , y_2 = (1,1)$

Theorem 2.-

Let $(E, \langle \cdot, \cdot \rangle)$ be a prehilbert space and $F \subseteq E$ a vectorial subspace, and $x \in E$ . Then :

a) If there is a better approximation from $x$ to $F$ , this is unique.

b) $y \in F$ is a best approximation from $x$ to $F$ $\iff$ $x - y$ is ortohogonal a $F$ .

Theorem 3.- (Gram- Schmidt) (1907 a. C.)

Let $(E, \langle \cdot, \cdot \rangle)$ be a prehilbert space and $\{x_1, ... ,x_n, ...\}$ a set of linearly independent vectors. If $y_1 := x_1,\quad u_1 = \frac{y_1}{|| y_1 ||}$ and $\displaystyle y_n := x_n - \sum_{j = 1}^{n -1} \langle x_n, u_j \rangle \space u_j, \quad u_n := \frac{y_n}{|| y_n ||}, \space n \ge 2.$ Then, $\{u_1, ... , u_n\}$ is a orthonormal ( and linearly independent) set $\forall n \in \mathbb{N}$ , and $\text{ Span} \{x_1, ... , x_n\} = \text{ Span} \{u_1, ... , u_n\}$

---

Solution of this problem

Let $\mathcal{C}([-1,1]) = \{ f: [-1, 1] \to \mathbb{R}; \text{f is continuous}\}$ be the vectorial space of the continuous functions defined on $[- 1, 1]$ over $\mathbb{R}$ whith the inner product (it's left to the reader that it is an inner product) $\langle f(t), g(t) \rangle = \int_{-1}^1 f(t)\cdot \overline{g(t)} \, dt = \int_{-1}^1 f(t)\cdot g(t) \, dt$ because $g(t)$ is a real continuous function. The problem is to find the best approximation from $x^3$ to the vectorial subspace $P_2 ([-1 ,1])$ of the polynomials of second grade having values on $[-1, 1]$ over $\mathbb{R}$ .I'm going to use Gram Schmidt theorem and theorem 2.

(Of course, there are more solutions, for instance, one quick solution is to take Legendre's polynomials(this solution is direct) or using Chebysev theorem or minimax method... or @Pi Han Goh proof, etc.).

$\{1, t, t^2\}$ is a basis of $P_2 ([-1 ,1])$ , and $\{1, t, t^2, t^3, ... \}$ is a basis of $P ([-1 ,1])$ (vectorial space of the polynomials..) $y_1 = 1, \langle 1, 1 \rangle = \int_{-1}^1 1 \, dt = 2 \Rightarrow u_1 = \frac{\sqrt{2}}{2}$ $y_2 = t - \int_{-1}^1 \frac{\sqrt{2}}{2}t \, dt \space \frac{\sqrt{2}}{2} = t - (1/2)\left[\frac{t^2}{2}\right]_{-1}^1 = t, \space \langle t, t \rangle = 2/3 \Rightarrow u_2 = \sqrt{3/2}t$ $y_3 = t^2 - (1/2)\int_{-1}^1 t^2 \, dt - \frac{3}{2}t \cdot \int_{-1}^1 t^3 \, dt = t^2 - \frac{1}{3} \Rightarrow u_3 = \frac{y_3}{|| y_ 3 ||}$ And finally, we just only need $y_4 = t^3 - (1/2)\int_{-1}^1 t^3 \, dt - \frac{3}{2}t \cdot \int_{-1}^1 t^4 \, dt - \frac{\langle t^3, y_3 \rangle}{\langle y_3, y_3 \rangle}y_3 = t^3 - (3/5)t$ , i.e, the unique best approximation from $x^3$ to $P_2 ([-1 ,1])$ is $(3/5)x$ and this means $\displaystyle \min_{a,b,c \in \mathbb{R}} \int_{-1}^1 |x^3 - a - bx - cx^2|^2 \, dx = \int_{-1}^1 (t^3 - (3/5)t)^2 \, dt = \frac{8}{175}$ .

Details.-

$\{1, t, t^2, t^3\}$ is a basis of $P_3([-1,1]) = \{\text{ set of poynomials at most of third degree on [-1, 1] over } \mathbb{R}\}$ , due to Gram-Schmidt $\{y_1, y_2 , y_3, y_4\}$ is an orthogonal basis of $P_3([-1,1]$ and $y_4 = x^3 - (3/5)x$ is orthogonal to $P_2([-1,1]$ with this inner product. Because of the problem is to find the best approximation from $x^3$ to $P_2([-1,1]$ , using theorem 2, it's not necessary to calculate $u_4$ or to multiply $y_4$ by a real number...

---

Theorem 1.- Cauchy- Schwartz inequality Generalisation

Let $(E, \langle \cdot, \cdot \rangle)$ be a prehilbert space. Then:

a) $\forall x, y \in E$ , $|\langle x ,y \rangle|^2 \leq \langle x ,x \rangle \cdot \langle y ,y \rangle$ (Cauchy-Schwartz inequality)

b) We can define a norm in $E$ . Thus,If $|| x || = + \sqrt{\langle x ,x \rangle}, \space \forall x \in E$ , then $|| \cdot ||$ is a norm

Proof of theorem 1.-

a) $\forall x, y \in E$ and $\lambda \in K = \mathbb{R}$ or $\mathbb{C}$ , $0 \leq \langle x - \lambda y, x - \lambda y \rangle = \langle x, x \rangle - \overline{\lambda} \langle x , y \rangle - \lambda \langle y , x \rangle + |\lambda|^2 \langle y , y \rangle;$ $\langle y , x \rangle = |\langle y , x \rangle| \cdot e^{i\theta} = b \cdot e^{i\theta}$ . Let $\lambda = t \cdot e^{-i\theta}$ with $t \in \mathbb{R}$ . Then $0 \leq \langle x , x \rangle - t \cdot e^{i\theta} \cdot b \cdot e^{-i\theta} - t \cdot e^{-i\theta} \cdot b \cdot e^{i\theta} + t^2 \langle y, y \rangle = \langle x , x \rangle - 2tb +t^2 \langle y , y \rangle, \space \forall t \in \mathbb{R}$ For this to happen $\forall t \in \mathbb{R}$ , the discriminant of this quadratic equation in $t$ , must be less than or equal to $0$ . This means $4b^2 - 4 \langle y, y \rangle \cdot \langle x, x \rangle \leq 0 \Rightarrow b^2 - \langle y, y \rangle \cdot \langle x, x \rangle \leq 0 \Rightarrow |\langle x, y \rangle|^2 \leq \langle y, y \rangle \cdot \langle x, x \rangle$ $\square$ = q.e.d

b) ,If $|| x || = + \sqrt{\langle x ,x \rangle}, \space \forall x \in E$ , then $|| \cdot ||$ is a norm.

i) $|| x || = + \sqrt{\langle x ,x \rangle} \ge 0, \space \forall x \in E \iff || x ||^2 = \langle x ,x \rangle \ge 0, \space \forall x \in E$ . (True)

$|| x || = + \sqrt{\langle x ,x \rangle} = 0 \iff || x ||^2 = \langle x ,x \rangle = 0 \iff x = 0$

ii) $||\lambda x||^2 = |\lambda|^2 \cdot || x ||^2 \Rightarrow ||\lambda x|| = |\lambda| ||x||, \space \forall x \in E$

iii) $|| x + y||^2 = \langle x , x \rangle + \langle x , y \rangle + \langle y , x \rangle + \langle y , y \rangle = ||x||^2 + 2 \mathfrak R \langle x , y \rangle + || y||^2 \leq$ $||x||^2 + 2 |\langle x , y \rangle| + || y||^2 \leq$ Applying Cauchy Schwartz inequality $||x||^2 + 2 ||x|| \cdot ||y|| + ||y||^2 = (||x|| + ||y||)^2$ Therefore, $|| x + y ||^2 \leq (||x|| + ||y||)^2 \Rightarrow ||x + y|| \leq ||x || + ||y||$ This last inequality is sometimes called Minkowsky inequality $\square$ (End of the proof of theorem 1)

Theorem 2.-

Let $(E, \langle \cdot, \cdot \rangle)$ be a prehilbert space and $F \subseteq E$ a vectorial subspace, and $x \in E$ . Then :

a) If there is a better approximation from $x$ to $F$ , this is unique.

I'll use Pythagoras's theorem : If $x \bot y$ then, $||x + y ||^2 = || x ||^2 + || y ||^2$ .

$||x + y ||^2 = \langle x + y, x + y \rangle = \langle x, x \rangle + \langle y, x \rangle + \langle x, y \rangle + \langle y, y \rangle = || x ||^2 + || y ||^2$ , $\square = \text{ q.e.d}$

Proof of a).- Suppose that we have just proved b)

Reductio ad absurdum. Suppose that $y_1$ and $y_2$ are two best approximattions from $x$ to $F$ , then $y_1 \neq y_2 \Rightarrow ||y_1 - y_2|| > 0, \Rightarrow ||x - y_1 ||^2 = ||x - y_2 + y_2 - y_1||^2 = || x - y_2 ||^2 + || y_2 - y_1||^2 > ||x - y_2||^2$ (Contradiction) $\square$

b) $y \in F$ is a best approximation from $x$ to $F$ $\iff$ $x - y$ is ortohogonal a $F$

$\Rightarrow$ ) Let $z \in F$ be any vector with $|| z || = 1$ , then define $w = y + \langle x - y, z \rangle \cdot z \in F$ . Because of $y$ is a best approximattion from $x$ to $F$ , $|| x - y ||^2 \leq ||x - w ||^2 = \langle x - y - \langle x - y, z \rangle \cdot z, x - y - \langle x - y, z \rangle \cdot z \rangle =$ $= ||x - y ||^2 - \langle x - y, z \rangle \cdot \langle z, x - y\rangle - \overline{\langle x -y, z \rangle} \langle x - y, z \rangle + |\langle x - y, z \rangle|^2 \cdot || z ||^2 =$ Using $|| z || = 1$ $= ||x - y ||^2 - |\langle x - y, z \rangle|^2 - |\langle x - y, z \rangle|^2 + |\langle x - y, z \rangle|^2 = || x - y ||^2 - |\langle x - y, z \rangle|^2 \Rightarrow |\langle x - y, z \rangle|^2 = 0$ $\square$

$\Leftarrow$ ) Let $z \in F$ be any vector , then $y - z \in F$ and $||x - z ||^2 = || x - y + y - z||^2 =$ applying Pythagoras's theorem $|| x - y||^2 + ||y - z||^2 \ge ||x - y ||^2 \Rightarrow$ y is the best approximation from $x$ to $F$ $\square$ (End of the proof of theorem 2)

Theorem 3.- (Gram- Schmidt) (1907 a. C.)

Let $(E, \langle \cdot, \cdot \rangle)$ be a prehilbert space and $\{x_1, ... ,x_n, ...\}$ a set of linearly independent vectors. If $y_1 := x_1,\quad u_1 = \frac{y_1}{|| y_1 ||}$ and $\displaystyle y_n := x_n - \sum_{j = 1}^{n -1} \langle x_n, u_j \rangle \space u_j, \quad u_n := \frac{y_n}{|| y_n ||}, \space n \ge 2.$ Then, $\{u_1, ... , u_n\}$ is a orthonormal ( and linearly independent) set $\forall n \in \mathbb{N}$ , and $\text{ Span} \{x_1, ... , x_n\} = \text{ Span} \{u_1, ... , u_n\}$

Proof of theorem 3.-

By induction:

1.- n=1 is trivial, supposse this result is true for $n \ge 1 \in \mathbb{N}$ , we are going to prove that this result is true for $n + 1$

2.- $y_{n + 1} = x_{n + 1} - \sum_{j = 1}^{n } \langle x_{n + 1}, u_j \rangle \space u_j \Rightarrow \langle y_{n + 1}, u_j \rangle =$ $= \langle x_{n + 1}, u_j \rangle - \langle x_{n + 1}, u_j \rangle = 0, \space \forall j \in \{1,2, ... , n\}$ due to $\{u_1, u_2, ... , u_n\}$ is an orthonormal set. This implies that $\{u_1, ... , u_{n + 1}\}$ is an orthonormal set and $\text{ Span} \{x_1, ... , x_n\} = \text{ Span} \{u_1, ... , u_n\}$ because $u_{n + 1} \in \text{ Span} \{x_1, ... , x_{n + 1}\}$ and $x_{n + 1} \in \text{ Span} \{u_1, ... , u_{n + 1}\}$ $\square$

---

Now, I'm going to give a corollary of this theorem, and other proof similar to the first solution.

Corollary of theorem 3 .-

Let $(E, \langle \cdot, \cdot \rangle)$ be a prehilbert space and $M \subseteq E$ a finite vectorial subspace. Then $\forall x \in E$ , there exists the best approximation $P_n (x)$ from $x$ to $M$ . If $\{u_1, u_2, ... , u_n\}$ is an orthonormal basis of $M$ , then $\displaystyle P_n (x) = \sum_{i = 1}^n \langle x, u_i \rangle \cdot u_i$ and $\displaystyle \text{ d(x, M)}^2 = || x ||^2 - \sum_{i = 1}^n | \langle x, u_i \rangle |^2$

Solution 2 of this problem based on previous solution and corollary

\displaystyle \min_{a, b, c \in \mathbb{R}} \int_{-1}^1 |x^3 - a - bx - cx^2|^2 \, dx = \text{ d(x^3, P\_2 ([-1, 1])}^2 = \int_{-1}^1 x^6 \, dx - |\int_{-1}^1 \sqrt{\frac{3}{2}} \cdot x^4 \, dx |^2 = $= \frac{2}{7} - \frac{6}{25} = \frac{50 - 42}{125} = \frac{8}{125},\space \square$

Proof of Corollary of theorem 3 .-

$\forall i = 1, 2, ... , n, \space \text{ fixed but arbitrary }, \space \langle x - P_n(x), u_i \rangle = \langle x , u_i \rangle - \langle x , u_i \rangle = 0 \Rightarrow$ $x - P_n(x) \bot \text{ Span } \{u_1, u_2, ... , u_n\} = M \Rightarrow$ $P_n (x)$ is the unique best approximation from $x$ to $M$ due to theorem 2.

Then, $\displaystyle \text{ d(x, M)}^2 = \langle x - P_n(x), x - P_n(x) \rangle = \langle x - P_n(x), x \rangle =$ $= || x ||^2 - \sum_{i = 1}^n | \langle x, u_i \rangle |^2, \square$

This theory will be applied and continued in this problem