Orthogonality and The Gram-Schmidt Process.
When it comes to working with vector sets, orthogonality makes things a whole lot easier, so in this article I will be talking about a cool trick to convert any vector set into an orthogonal one. First we need to understand a little about orthogonal vectors and inner products, so lets start with that.
Orthogonal/Orthonomral vectors:
Orthogonal vectors are defined as 2 vectors that are perpendicular to eachother, or in other words at a 90* angle:
and if you were to perform a dot product analysis on them, the product they give would be zero:
An Orthonormal vector on the other hand is defined as a unit vector of length 1. Say we have a vector u, in order to determine if u is a unit vector we use the equation below:
Dividing vector u by |u| (its length, also reffered to as its norm), gives you its unit vector u_hat. If equal to 1, then vector u is a unit vector and is considered orthonormal.
Orthonormal Matrix:
Orthnormal Matrix is defined as a set of vectors that are all mutually orthogonal to eachother, and all have a unit vector lenght of 1. Futhermore, say we have a matrix Q, and the following applies to it, we can then say that:
This means that matrix Q is orthonormal if its transpose is equal to its inverse:
And the determinant of an orthonormal matrix is either +1 or -1.
Working with an orthonormal basis makes are life much easier when it comes to vectors and matrices, getting things like its inverse, transpose and determinant are much simpler.
Inner Product:
An inner product space is a real vector space or a complex vector space with an operation called inner product, see below:
Where the dot product of x and y divided by the multiple of the length of x and y return the cosine value, or angle of two vectors in scalar form. If x and y are orthogonal, then the cosine value is equal to 0.
Gram-Shmidt Process:
The Gram-Shmidt Process allows us to transform a set of linearly independent vectors into a set of orthogonal vectors forming an orthonormal basis. If your unsure what linear independence means, refer here.
Lets first look at the projection operator, which is defined like this:
*<v,u> denotes the inner product of the vectors u and v.
This operator projects the vector v orthogonally onto the line spaned by vector u, and if u = 0, then we can define proj(v) := 0, or in other words the projection map “proj()” is the zero map, sending every vector to the zero vector.
Through understanding the above, we can now look at the Gram-Shmidt process, which works as follows:
Notice we run through all the vectors u, and at the same time create a new vector e. In therory we are sequencing through u, and creating a new vector set e which are an orthonormal set representing u. This calculation in the sequence u is whats refferred to as Gram-Schmidt orthogonalization, while the calcualation sequence e is known as Gram-Schmidt orthonormalization as the vectors are normalized.
The best way to understand this is by seeing how its done through a set of vectors graphed out like so:
If we zoom in further it would look like this:
Techinically speaking, we would first need to make sure these vectors are linearly indepndent from eachother, but in this case lets assume they are. With the red line as vector v1 and the green as vector v2, first we make v1 temporarily equal to u1:
Now that the red line is considered vector u1, we can find the projection of v2 onto u1, and as stated at the beginning of the section, we calculate the projection of v2 with the equation below:
After we find the projection, we continue with the Gram-Schmidt process:
*Notice, we’re subtracting the projection of v2 from itself, resulting in our new vector u2 (transformed v2) having its projection onto u1 equal to 0.
After the above transformation is complete, the red and green line are now orthogonal, and our entire vector set would look something like this:
Following the gram-schmidt process, the next equation in the process is below:
Notice that the above equation is a two part process, we need the projection of v3 onto both u1 and u2, and then subtract both of them with v3. In order to calculate this succesfully we need to do one projection at a time, so lets start by getting the projection of v3 onto u1 first:
then we subtract v3 with its u1 projection:
After that, we can now move onto the second vector u2, first getting v3’s projection onto it:
And subtracting v3 by its u2 projection:
*Important: As stated before, its important that you do one projection and subtraction at a time, getting v3’s projection on multiple vectors at the same time and then subtracting will not give you an orthogonal set. In our case we made v1 and v2 orthogonal, then v3 and v1, and finally v3 and v1 respectively.
If there is more vectors in the set, we simply continue the process above, getting the projection of one vector onto another and subtracting it for each vector in the set.
After all this, we get the following orthogonal vector set:
We finish off by getting the unit vector lenghts of the new vectors:
If there orthonormal, they should all be equal to one, like below:
We’ve now taken a set of linearly independent vectors, and converted it to an orthogonal set with an orthonomral basis!
Conclusion:
In this article, we’ve found that in order to convert a linearlly independent vector set into an orthonormal basis, we simply use the Gram-schmidt process, which consists of first getting the vector projection of one vector onto another, then subtracting it by by its projection, and repeating this process with all other vectors until the vector set is entirely orthogonal, finally getting the unit vector of all the vectors converting it into an orthonormal basis. I hope this was helpful, and if you notice any errors feel free to make a comment!
Sources:
