Lab 4: Orthogonality and Projections

due by the end of your lab section on Wednesday, September 23rd, 2026

Each lab worksheet will contain several activities, some of which will involve writing code and others that will involve writing math on paper. To receive credit for a lab, you must complete all activities and show your lab TA by the end of the lab section. To receive credit for Activity 1, you’ll need to show your lab TA that all test cases have passed and that you have answered the written questions in Task 4. Instructions on how to do this are in the lab notebook.

While you must get checked off by your lab TA individually, we encourage you to form groups with 1-2 other students to complete the activities together.

Activities

Activity 1: Presidential Speeches and Cosine Similarity
Activity 2: Sum-Difference Orthogonality
Activity 3: Orthogonal Projections
Activity 4: Orthogonal Decomposition with Orthonormal Vectors
Activity 5: Orthogonal Decomposition with Non-Unit Vectors
Activity 6: A Plane from Spanning Vectors

Recap: Projections

(Chapter 3.4) The orthogonal projection of the vector $\vec u$ onto the vector $\vec v$ is given by
$\vec p = \frac{\vec u \cdot \vec v}{\vec v \cdot \vec v} \vec v$
Above, the scalar $k^{\ast} = \frac{\vec u \cdot \vec v}{\vec v \cdot \vec v}$ was chosen to minimize $\lVert \vec u - k \vec v \rVert^2$ .
The vector $\vec p$ is called the orthogonal projection because the resulting error vector,
$\vec e = \vec u - \vec p = \vec u - k^* \vec v$
is orthogonal to $\vec v$ , meaning $\vec e \cdot \vec v = 0$ .

Activity 1: Presidential Speeches and Cosine Similarity

Complete the tasks in the lab04.ipynb notebook.

There are two ways to access the supplemental Jupyter Notebook:

Option 1 (preferred): Set up a Jupyter Notebook environment locally, use git to clone our course repository, and open labs/lab04/lab04.ipynb. For instructions on how to do this, see the Environment Setup page of the course website.
Option 2: Click here to open lab04.ipynb on DataHub. Before doing so, read the instructions on the Environment Setup page on how to use the DataHub.

To receive credit for Activity 1, you’ll need to show your lab TA that all test cases have passed and that you have answered the written questions in Task 4.

Activity 2: Sum-Difference Orthogonality

Let $\vec u=\begin{bmatrix}2\\-1\\0\\5\end{bmatrix}$ and $\vec v=\begin{bmatrix}1\\2\\4\\-3\end{bmatrix}$ .

Show that $\vec u+\vec v$ and $\vec u-\vec v$ are orthogonal.

Solution

Let’s start by computing the two vectors:

\vec u+\vec v=\begin{bmatrix}3\\\\1\\\\4\\\\2\end{bmatrix},\qquad \vec u-\vec v=\begin{bmatrix}1\\\\-3\\\\-4\\\\8\end{bmatrix}.

Their dot product is

(\vec u+\vec v)\cdot(\vec u-\vec v) =3\cdot1+1\cdot(-3)+4\cdot(-4)+2\cdot 8 =3-3-16+16 =0.

Since the dot product is $$0$$ , the vectors are orthogonal.

Now suppose $\vec u,\vec v\in\mathbb{R}^n$ are arbitrary vectors with the same number of components. Is it always true that $\vec u+\vec v$ and $\vec u-\vec v$ are orthogonal?

If so, prove why.
If not, specify conditions under which it’s guaranteed that $\vec u+\vec v$ and $\vec u-\vec v$ are orthogonal.

Hint: Use the distributive property of the dot product, which states that

(\vec a + \vec b) \cdot (\vec c + \vec d) = \vec a \cdot \vec c + \vec a \cdot \vec d + \vec b \cdot \vec c + \vec b \cdot \vec d

Solution

For any two vectors $\vec u, \vec v \in \mathbb{R}^n$ ,

(\vec u+\vec v)\cdot(\vec u-\vec v) = \vec u\cdot\vec u - \vec u\cdot\vec v + \vec v\cdot\vec u - \vec v\cdot\vec v = \|\vec u\|^2 - \|\vec v\|^2,

since $\vec u\cdot\vec v=\vec v\cdot\vec u$ .

So, in order for $\vec u+\vec v$ and $\vec u-\vec v$ to be orthogonal, we need

\|\vec u\|^2 - \|\vec v\|^2 = 0

which means

\|\vec u\| = \|\vec v\|

So, $\vec u+\vec v$ and $\vec u-\vec v$ are orthogonal if (and only if!) the two vectors have equal length. That was the case in part a) — both vectors had a norm of $\sqrt{2^2 + (-1)^2 + 0^2 + 5^2} = \sqrt{30}$ .

Activity 3: Orthogonal Projections

Let $\vec c = \begin{bmatrix} 1 \\ 2 \\ -4 \\ 0 \end{bmatrix}$ and $\vec d = \begin{bmatrix} 3 \\ 2 \\ 0 \\ -1 \end{bmatrix}$ .

Find the orthogonal projection of $\vec c$ onto $\vec d$ . Call this vector $\vec q$ .

Solution

\begin{align*} \vec q &= \left(\frac{\vec c \cdot \vec d}{\vec d \cdot \vec d}\right)\vec d \\\\ &= \frac{1 \cdot 3 + 2 \cdot 2 + (-4) \cdot 0 + 0 \cdot (-1)}{3^2 + 2^2 + 0^2 + (-1)^2} \vec d \\\\ &= \frac{3+4}{9+4+1} \vec{d} \\\\ &= \frac{7}{14} \vec{d} \\\\ &= \frac{1}{2} \vec{d} \\\\ &= \begin{bmatrix} 1.5 \\\\ 1 \\\\ 0 \\\\ -0.5 \end{bmatrix} \end{align*}

Find the error vector, $\vec r = \vec c - \vec q$ . Which vector is $\vec r$ orthogonal to, $\vec c$ or $\vec d$ ? Draw a rough picture of the relationship between $\vec c$ , $\vec d$ , $\vec q$ , and $\vec r$ . You may want to review Chapter 3.4.

Solution

First, let’s find $\vec{r}$ .

\begin{align*} \vec{r} &= \vec{c} - \vec{q} \\\\ &= \begin{bmatrix} 1 \\\\ 2 \\\\ -4 \\\\ 0 \end{bmatrix} - \begin{bmatrix} 1.5 \\\\ 1 \\\\ 0 \\\\ -0.5 \end{bmatrix} \\\\ &=\begin{bmatrix} -0.5 \\\\ 1 \\\\ -4 \\\\ 0.5 \end{bmatrix} \end{align*}

$\vec{r}$ is orthogonal to $\vec{d}$ , not $\vec{c}$ , as confirmed by the dot products. The key idea we introduced in Chapter 3.4 is that the error vector is orthogonal to the vector we projected onto. Here, $\vec r$ is the error vector and $\vec d$ is the vector we projected onto.

\begin{align*} \vec{r} \cdot \vec{c} &= (-0.5) \cdot 1 + 1 \cdot 2 + (-4) \cdot (-4) + 0.5 \cdot 0 \\\\ &= -0.5 + 2 + 16 \\\\ &= 17.5 \\\\ \\\\ \vec{r} \cdot \vec{d} &= (-0.5) \cdot 3 + 1 \cdot 2 + (-4) \cdot 0 + 0.5 \cdot (-1) \\\\ &= -1.5 + 2-0.5 \\\\ &=0 \end{align*}

Activity 4: Orthogonal Decomposition with Orthonormal Vectors

Let

\vec{v}_1 = \begin{bmatrix} \frac{1}{\sqrt{5}}\\\\[2pt] \frac{2}{\sqrt{5}} \end{bmatrix}, \qquad \vec{v}_2 = \begin{bmatrix} -\frac{2}{\sqrt{5}}\\\\[2pt] \frac{1}{\sqrt{5}} \end{bmatrix}

$\vec v_1$ and $\vec v_2$ are called orthonormal, because they are:

unit vectors: $\lVert \vec v_1 \rVert = 1$ and $\rVert \vec v_2 \rVert = 1$ .
orthogonal: $\vec v_1 \cdot \vec v_2 = 0$ .

Additionally, let $\vec u = \begin{bmatrix} 4 \\ -1 \end{bmatrix}$ .

Our goal in this activity is to write $\vec{u}$ as a linear combination of $\vec{v}_1$ and $\vec{v}_2$ . The fact that $\vec v_1$ and $\vec v_2$ are orthonormal makes this simple.

Find $\vec u \cdot \vec v_1$ and $\vec u \cdot \vec v_2$ . Your answer should involve $\sqrt{5}$ ; don’t use a calculator.

Solution

To find each dot product, we multiply corresponding components and add:

\begin{align*} \vec u \cdot \vec v_1 &= 4 \cdot \frac{1}{\sqrt{5}} + (-1) \cdot \frac{2}{\sqrt{5}} = \frac{4 - 2}{\sqrt{5}} = \boxed{\frac{2}{\sqrt{5}}} \\\\ \vec u \cdot \vec v_2 &= 4 \cdot \left(-\frac{2}{\sqrt{5}}\right) + (-1) \cdot \frac{1}{\sqrt{5}} = \frac{-8 - 1}{\sqrt{5}} = \boxed{-\frac{9}{\sqrt{5}}} \end{align*}

Evaluate

(\vec u \cdot \vec v_1) \vec v_1 + (\vec u \cdot \vec v_2) \vec v_2

What do you notice?

Solution

Using the dot products we found in part a), we have

\begin{align*} (\vec u \cdot \vec v_1)\vec v_1 + (\vec u \cdot \vec v_2)\vec v_2 &= \frac{2}{\sqrt{5}}\begin{bmatrix} \frac{1}{\sqrt{5}} \\\\ \frac{2}{\sqrt{5}} \end{bmatrix} - \frac{9}{\sqrt{5}}\begin{bmatrix} -\frac{2}{\sqrt{5}} \\\\ \frac{1}{\sqrt{5}} \end{bmatrix} \\\\ &= \begin{bmatrix} \frac{2}{5} \\\\ \frac{4}{5} \end{bmatrix} + \begin{bmatrix} \frac{18}{5} \\\\ -\frac{9}{5} \end{bmatrix} \\\\ &= \begin{bmatrix} 4 \\\\ -1 \end{bmatrix} = \boxed{\vec u} \end{align*}

We get back $\vec u$ ! In other words, the dot products from part a) are exactly the coefficients we need to write $\vec u$ as a linear combination of $\vec v_1$ and $\vec v_2$ .

Why is $(\vec u \cdot \vec v_1) \vec v_1 + (\vec u \cdot \vec v_2) \vec v_2 = \vec u$ ?

Hint: Start by drawing a picture of $\vec u$ , $\vec v_1$ , and $\vec v_2$ , and reviewing the “Orthogonal Decomposition” section of Chapter 3.4 and the very last example discussed in yesterday’s lecture.

Solution

The two terms in part b) are the projections of $\vec u$ onto the orthogonal directions $\vec v_1$ and $\vec v_2$ . Since these vectors are orthogonal, their projections add to $\vec u$ , just as in the orthogonal decomposition we saw in lecture.

Activity 5: Orthogonal Decomposition with Non-Unit Vectors

Let $\vec{v}_1 = \begin{bmatrix} -1 \\ 2 \\ 2 \end{bmatrix}$ $\vec{v}_2 = \begin{bmatrix} 2 \\ 2 \\ -1 \end{bmatrix}$ and $\vec{v}_3 = \begin{bmatrix} 2 \\ -1 \\ 2 \end{bmatrix}$ . Write $\vec{u} = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}$ as a linear combination of $\vec{v}_1$ , $\vec{v}_2$ , and $\vec{v}_3$ , and verify that your answer is correct. Note that $\vec v_1$ , $\vec v_2$ , and $\vec v_3$ are pairwise orthogonal.

Solution

We are given

\vec v_1=\begin{bmatrix}-1\\\\2\\\\2\end{bmatrix}\quad \vec v_2=\begin{bmatrix}2\\\\2\\\\-1\end{bmatrix}\quad \vec v_3=\begin{bmatrix}2\\\\-1\\\\2\end{bmatrix}\quad \vec u=\begin{bmatrix}1\\\\1\\\\1\end{bmatrix}

We’re looking for scalars $$a,b,c$$ such that $\vec u=a\vec v_1+b\vec v_2+c\vec v_3$ .

Solution 1: Solving a system of equations

a\!\begin{bmatrix}-1\\\\2\\\\2\end{bmatrix} +b\!\begin{bmatrix}2\\\\2\\\\-1\end{bmatrix} +c\!\begin{bmatrix}2\\\\-1\\\\2\end{bmatrix} =\begin{bmatrix}1\\\\1\\\\1\end{bmatrix}

Is equivalent to the system of equations:

\begin{align} -a+2b+2c=1 \\\\ 2a+2b-c=1 \\\\ 2a-b+2c=1 \end{align}

Using elimination:

\begin{align*} \text{(Eq.\,2)}-\text{(Eq.\,3)}&:\quad (2a-2a)+(2b-(-b))+(-c-2c)=0 \\\\[-2pt] &\qquad\Rightarrow\quad 3b-3c=0 \quad\Rightarrow\quad b=c \\\\[6pt] \text{(Eq.\,2)}-\text{(Eq.\,1)}&:\quad (2a-(-a))+(2b-2b)+(-c-2c)=0 \\\\[-2pt] &\qquad\Rightarrow\quad 3a-3c=0 \quad\Rightarrow\quad a=c \end{align*}

This tells us that $$a = b = c$$ . Plugging this back into Eq. 2 gives us:

-a + 2a + 2a = 1 \rightarrow 3a = 1 \rightarrow a = \frac{1}{3}

So, $a = b = c = \frac{1}{3}$ , and:

\vec u=\tfrac13\,\vec v_1+\tfrac13\,\vec v_2+\tfrac13\,\vec v_3

We can verify that we did this correctly by computing the right-hand side above:

\frac{1}{3}\vec v_1+\frac{1}{3}\vec v_2+\frac{1}{3}\vec v_3 =\frac{1}{3}\begin{bmatrix}-1\\\\2\\\\2\end{bmatrix}+\frac{1}{3}\begin{bmatrix}2\\\\2\\\\-1\end{bmatrix}+\frac{1}{3}\begin{bmatrix}2\\\\-1\\\\2\end{bmatrix} =\begin{bmatrix}-1/3+2/3+2/3\\\\2/3+2/3-1/3\\\\2/3-1/3+2/3\end{bmatrix} =\begin{bmatrix}1\\\\1\\\\1\end{bmatrix} =\vec u

Solution 2: Using the fact that $\vec v_1, \vec v_2, \vec v_3$ are orthogonal

As is alluded to in part b), we can use the fact that $\vec v_1, \vec v_2, \vec v_3$ are orthogonal to find coefficients $$a$$ , $$b$$ , and $$c$$ by projecting $\vec u$ onto each of the $\vec v_i$ s. This is similar to what was done in the Orthogonal Decomposition section of Chapter 3.4.

Let $\vec p_i$ be the projection of $\vec u$ onto $\vec v_i$ , for $$i = 1, 2, 3$$ . Then, we have:

\vec p_1 = \frac{\vec u \cdot \vec v_1}{\vec v_1 \cdot \vec v_1} \vec v_1 = \frac{1 \cdot (-1) + 1 \cdot 2 + 1 \cdot 2}{(-1)^2 + 2^2 + 2^2} \vec v_1 = \frac{3}{9} \vec v_1 = \frac{1}{3} \vec v_1

\vec p_2 = \frac{\vec u \cdot \vec v_2}{\vec v_2 \cdot \vec v_2} \vec v_2 = \frac{1 \cdot 2 + 1 \cdot 2 + 1 \cdot (-1)}{2^2 + 2^2 + (-1)^2} \vec v_2 = \frac{3}{9} \vec v_2 = \frac{1}{3} \vec v_2

\vec p_3 = \frac{\vec u \cdot \vec v_3}{\vec v_3 \cdot \vec v_3} \vec v_3 = \frac{1 \cdot 2 + 1 \cdot (-1) + 1 \cdot 2}{2^2 + (-1)^2 + 2^2} \vec v_3 = \frac{3}{9} \vec v_3 = \frac{1}{3} \vec v_3

Adding $\vec p_1, \vec p_2, \vec p_3$ gives us:

\vec p_1 + \vec p_2 + \vec p_3 = \frac{1}{3} \vec v_1 + \frac{1}{3} \vec v_2 + \frac{1}{3} \vec v_3 = \vec u

In general, suppose that $\vec v_1, \vec v_2, \ldots, \vec v_d$ are orthogonal vectors in $\mathbb{R}^n$ , meaning that $\vec v_i \cdot \vec v_j = 0$ for all $i \neq j$ . Given that it is possible to write $\vec u$ as a linear combination of $\vec v_1, \vec v_2, \ldots, \vec v_d$ ,

show that the coefficients of the linear combination

\vec u = a_1 \vec v_1 + a_2 \vec v_2 + \cdots + a_d \vec v_d

are given by

a_i = \frac{\vec u \cdot \vec v_i}{\vec v_i \cdot \vec v_i}

Hint: Start by taking the dot product of both sides of the linear combination equation with $\vec v_1$ . What do you notice?

Solution

We’re told to assume that any pair of vectors among $\vec v_1, \vec v_2, \ldots, \vec v_d$ are orthogonal, and that $\vec u$ can be written as a linear combination of $\vec v_1, \vec v_2, \ldots, \vec v_d$ .

\vec u = a_1 \vec v_1 + a_2 \vec v_2 + \cdots + a_d \vec v_d

As the hint suggests, let’s take the dot product of both sides with $\vec v_i$ , where $$i$$ is some value in $\lbrace1, 2, \ldots, d\rbrace$ .

\vec u\cdot \vec v_i =\bigl(a_1\vec v_1+\cdots+a_d\vec v_d\bigr)\cdot \vec v_i

Since $\vec v_i\cdot \vec v_j=0$ for $i\neq j$ , only the $$i=j$$ term survives:

\begin{align*} \vec u\cdot \vec v_i &= \bigl(a_1\vec v_1+\cdots+a_d\vec v_d\bigr)\cdot \vec v_i \\\\[6pt] &= a_1(\vec v_1\cdot \vec v_i)+\cdots+a_{i-1}(\vec v_{i-1}\cdot \vec v_i) +a_i(\vec v_i\cdot \vec v_i) +a_{i+1}(\vec v_{i+1}\cdot \vec v_i)+\cdots+a_d(\vec v_d\cdot \vec v_i) \\\\[6pt] &= a_1(0)+\cdots+a_{i-1}(0) +a_i(\vec v_i\cdot \vec v_i) +a_{i+1}(0)+\cdots+a_d(0) \\\\[6pt] &= a_i(\vec v_i\cdot \vec v_i) \\\\[6pt] \end{align*}

Solving for $$a_i$$ above gives us

\vec u \cdot \vec v_i = a_i(\vec v_i\cdot \vec v_i) \implies a_i = \frac{\vec u \cdot \vec v_i}{\vec v_i\cdot \vec v_i}

Since $$i$$ was arbitrary, the same calculation holds for any value of $$i$$ in $\lbrace1, 2, \ldots, d\rbrace$ .

What we proved here in part b) is that when writing a vector $\vec u$ as a linear combination of orthogonal vectors, the coefficients of the linear combination can be found by projecting the vector $\vec u$ onto each of the orthogonal vectors and adding the results, rather than solving a system of equations.

In Activity 4, why didn’t we divide by $\vec v_1 \cdot \vec v_1$ (and $\vec v_2 \cdot \vec v_2$ ) when finding the coefficients?

Solution

The vectors in Activity 4 are unit vectors, so $\vec v_1 \cdot \vec v_1 = \lVert \vec v_1 \rVert^2 = 1$ , and likewise for $\vec v_2$ . Dividing by these dot products would just mean dividing by 1.

Activity 6: A Plane from Spanning Vectors

This activity previews ideas that we will explore in upcoming lectures.

An important idea from Chapter 4.1 is that two non-parallel vectors in $\mathbb{R}^n$ (where $n \geq 2$ ) span a plane in $$n$$ -dimensional space. Here, we’ll show you how to find the equation of such a plane, given two vectors in $\mathbb{R}^3$ . This is also touched on in Chapter 4.4.

Let

\vec u = \begin{bmatrix} 5 \\\\ -7 \\\\ 3 \end{bmatrix}, \qquad \vec v = \begin{bmatrix} 4 \\\\ 1 \\\\ -2 \end{bmatrix}, \qquad P = \operatorname{span}(\{\vec u, \vec v\})

Find a nonzero vector $\vec n$ that is orthogonal to both $\vec u$ and $\vec v$ . Verify your answer using dot products.

Solution

Write $\vec n = \begin{bmatrix} a \\ b \\ c \end{bmatrix}$ . Orthogonality requires

5a - 7b + 3c = 0, \qquad 4a + b - 2c = 0.

The second equation gives $$b = 2c - 4a$$ . Substituting into the first gives $$33a - 11c = 0$$ , so $$c = 3a$$ and $$b = 2a$$ .

Note that there are infinitely many choices of $$a$$ , meaning there are infinitely many $\vec n$ that satisfy the constraints of the question. All we were asked for is a nonzero vector that is orthogonal to both $\vec u$ and $\vec v$ ; once we find one such vector, any scalar multiple of it (of which there are infinitely many!) will also be orthogonal to both $\vec u$ and $\vec v$ . Since we just need one vector, let’s keep it simple: pick $$a = 1$$ . Then,

\boxed{\vec n = \begin{bmatrix} 1 \\\\ 2 \\\\ 3 \end{bmatrix}}.

Indeed, $\vec n \cdot \vec u = 5 - 14 + 9 = 0$ and $\vec n \cdot \vec v = 4 + 2 - 6 = 0$ .

Suppose your vector is $\vec n = \begin{bmatrix} a \\ b \\ c \end{bmatrix}$ . Verify that both $\vec u$ and $\vec v$ satisfy

$$ ax + by + cz = 0 $$

Then, graph your equation on Desmos, desmos.com/3d.

Solution

Using $\vec n = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$ from part a), the equation is

\boxed{x + 2y + 3z = 0}.

Substituting the components of $\vec u$ gives $$5 + 2(-7) + 3(3) = 0$$ , and substituting the components of $\vec v$ gives $$4 + 2(1) + 3(-2) = 0$$ . Thus, both vectors satisfy the equation. The graph in Desmos is the plane through the origin spanned by $\vec u$ and $\vec v$ .