Orthogonal

more to say

1. you see | number |, you see || vector ||

notation of $x^Ty$

another way to denote inner prodoct, i.e. dot product

def. of euclidean length

distance from origin to arror end. Sum of each component’s square and take square root

def. of distance

distance of arbitary two vector

scalar product of vector length

simply times a scalar

def. $\perp$ vector in $\mathbb{R}^n$

inner product is zero

勾股定理

same in geometric

def. of projection with dir. vector

$\text{the projection of v on to u,} Proj_u(v)=\frac{v\cdot u}{||u||}$

$Proj_u(v) = \frac{v\cdot u}{u\cdot u}$

orthogonal subspace in $\mathbb{R}^n$

any vector from V denote as v, similar W. Have property that $v\cdot w=0$ for all v in V and w in W.
a plane and its perp line

Orthogonal complement and its properties

Fundamental subspace theorem

$dim S + dim S^\perp = n$   $(S \in \mathbb{R}^n)$

def. of orthogonal $\perp$

Lemma 19.5 (Cauchy-Schwartz Inequality)

$|x^Ty| ≤ ||x||.||y||$


$\mathbf{a}\cdot\mathbf{b} = 0||\vec a |$

$x^Ty = 0$

least quare solution

normal equations
最小二乘

$\begin{bmatrix} 9 & -7^2 \\ -7 & 11_1 \end{bmatrix}$

$prop$

$1. A^TA\hat{x} = A^T\mathbf{b} \\ 2. A^TA\hat{x} = \mathbf{b} \\ 3. ||A^TA\hat{x} - \mathbf{b}||^2 = ||A^T\mathbf{b} - \mathbf{b}||^2 \\ 4. alpha = \frac{||A^TA\hat{x} - \mathbf{b}||^2} 5.$

inner product in more general way

denote as $<x,y>$, x and y are vector in vector space V, to be more particular, called it “inner product space”.
general inner product must fufill 3 properties as follows:

$(1)<x,y> = <y,x> \\ (2)<ax+by,z> = a<x,z> + b<y,z> \{a,b\in \mathbb{R} \text{ and } x,y,z \in V\}\\ (2.1)<x+y,z> = <x,z> + <y,z> \\ (2.2)<cx,y> = c<x,y>\\ (3)<x,x> \geq 0 \text{ and equality holds iff. } x = 0$

Frobenius inner product (inner product for matrix)

definition
For matrices $A$ and $B$ of the same dimensions, the Frobenius inner product is defined as:

$<A, B>_F = \mathrm{tr}(A^TB)$

Where $\mathrm{tr}(\cdot)$ is the trace of a matrix.

这个看不懂，但是可以写成矩阵相同位置一一相乘的形式

length in inner product space

a vector v in inner product space, length of v is denote as

$||v|| = \sqrt{<v,v>}$

orthorgonal in inner product space

seems that we define a new way for inner produc
in previews learning, we define inner product as $x^Ty$, which is traditional inner product taught in high school
but now inner product can be define in another way
so correspondingly we have a more general definition of what inner product is. And what orthogonal is.(since it is defined as <x,y> = 0)

Theo. 勾股定理 in inner VS

垂直有0，产生的等式

Theo. 21.5 Cauchy inequality

$|<x,y>| \geq ||x||\cdot||y||$

normed vector space

A vector space V is said to be a normed linear space if

each vector v ∈ V is associated with a real
number ||v||∈ R, called the norm of v, satisfying:

$1. ||cv|| = |c|\cdot||v|| \\ 2. v \geq 0, = when v = 0 \\ 3. \text{triangle inequality}$

Theo 21.7 norm on inner product space

Orthogonal set

for $\{v_1, \dotsc, v_n\}$ in inner product space V, any $v_i \not ={v_j}$, that $v_i \cdot v_j = 0$, then the set is orthogonal set

Orthonomal set

in Orthogonal set, take all the vactor with norm == 1, then it is orthonormal set.

Theo. 21.10 Orthogonal set is linear indep.

def. orthonomal basis

1. B is the basis if B is an orthonomal set of V
2. Span{B} = V

coor. w.r.t orthonomal basis

any V can be decompose

def. 21.25 orthogonal matrix

if column vector in Q is orthonomal set in $\mathbb{R}^n$, then Q is orthogonal matrix
*好严苛的判定

equivalent condition for orthogonal matrix

$Q^{-1} = Q^T$

prop. 5 properties that orthogonal matrix have

$1. QQ^T=Q^TQ=I\\ 2. det(Q)=±1\\ 3. \text{The columns of Q form an orthonormal basis of }R^n\\ 4. Q^{-1}=Q^T\\ 5. If A is symmetric, then Q^TAQ is also symmetric\\ 6. ||Qx|| = ||x||$

Gram Schmidt Process 格拉姆-施密特

对 $R^n$ 中任意非零子空间，快速求解，正交基/标准正交基 的方法