(4 pts) Find a nonzero vector \(\vec n\) orthogonal to both \(\vec v_1\) and \(\vec v_2\), and use it to find an equation for the plane \(S\) of the form \(ax+by+cz=0\).
Homework 5: Matrices
due Sunday, October 18th, 2026 at 11:59PM Ann Arbor Time
(Homeworks are now due on Sundays!)
Except for the video link in Problem 1, write your solutions to the following problems either by writing them on a piece of paper or on a tablet and scanning your answers as a PDF. Note that you are not allowed to use LaTeX, Google Docs, or any other digital document creation software to type your answers. Homeworks are due to Pensive by 11:59PM on the due date. See the syllabus for details on the slip day policy.
Homework will be evaluated not only on the correctness of your answers, but on your ability to present your ideas clearly and logically. You should always explain and justify your conclusions, using sound reasoning. Your goal should be to convince the reader of your assertions. If a question does not require explanation, it will be explicitly stated.
Before proceeding, make sure you’re familiar with the collaboration policy.
Problems
- Problem 1: Midterm 1 Video Reflection
- Problem 2: Projecting onto a Plane
- Problem 3: Getting Started with Matrices
- Problem 4: Correlation, Revisited
- Problem 5: Projections, Revisited
- Problem 6: Orthogonal Matrices
- Problem 7: Rank and CR Decomposition
- Problem 8: Random Matrices, Rank, and Cosine Similarity
Total Points: 5 + 12 + 15 + 10 + 20 + 20 + 10 + 8 = 100
Note: You may find it helpful to check some of your answers using numpy, once you’ve already done the calculations by hand. This video gives you tips on using numpy for various matrix calculations. It’s important to learn how to do both – doing calculations by hand, so you know how they actually work, and using code to perform computation efficiently.
Problem 1: Midterm 1 Video Reflection (5 pts)
Instead of asking you to write a reflection, you’ll need to record a video !
Review the solutions to Midterm 1. Pick one question or problem part that you answered incorrectly, but now understand after reviewing the solutions and reflecting on your work.
Record a 2- to 5-minute video explaining that question. Tell us what the question asks, what you got wrong or misunderstood, and how to solve it now. Explain the reasoning in your own words; don’t just read the posted solution aloud.
The video should be just you speaking into the camera. Your face must be visible while you are talking for the entire video. Do not include slides, a screen recording, written work, or any other visuals.
Submit your video link to the separate assignment on Pensive called Homework 5, Problem 1 Video Link. You can upload your video to YouTube as unlisted, share a Zoom recording link, or share a Google Drive link; whatever works. Make sure the course staff can view the video using that link.
Your submission time for Homework 5 is the later of the two submission times: your Problem 1 video link and your regular Homework 5 PDF.
This problem is graded on completion. Record a video of the required length and make an honest effort to explain the question, and you will receive full credit. Your explanation does not need to be polished or perfect.
Problem 2: Projecting onto a Plane (12 pts)
In Lecture 11, we stated our eventual goal: projecting \(\vec u\) onto \(\operatorname{span}(\lbrace\vec v_1,\ldots,\vec v_d\rbrace)\). We already have enough information for a special case: projecting onto a plane through the origin in \(\mathbb R^3\).
Let
$$ \vec v_1=\begin{bmatrix}2\\\\3\\\\-1\end{bmatrix},\qquad \vec v_2=\begin{bmatrix}3\\\\4\\\\-2\end{bmatrix},\qquad \vec u=\begin{bmatrix}4\\\\1\\\\0\end{bmatrix} $$
Let \(S=\operatorname{span}(\lbrace\vec v_1,\vec v_2\rbrace)\).
a)
b)
(3 pts) Find \(\vec q\), the projection of \(\vec u\) onto \(\vec n\). Then compute \(\vec p=\vec u-\vec q\).
c)
(5 pts)
Verify that \(\vec p\) is on the plane from part a).
Write \(\vec p\) as a linear combination of \(\vec v_1\) and \(\vec v_2\).
Explain why \(\vec p\) is the projection of \(\vec u\) onto \(S\).
Problem 3: Getting Started with Matrices (15 pts)
Let
$$ A = \begin{bmatrix} 3 & 0 & 4 \\\\ 0 & 1 & 0 \\\\ 2 & -1 & -3 \\\\ 5 & 0 & -1 \\\\ 3 & 2 & 0 \end{bmatrix} $$
a)
(4 pts) In each subpart, state whether the resulting object is a matrix, vector, or scalar. If the result is a matrix or vector, state its dimensions. If the result is not defined, state why. You don't need to actually compute the resulting objects.
\(A^T\)
\(A^TA\)
\(AA^T\)
\(A^TA + AA^T\)
\(A^T \vec x\), where \(\vec x \in \mathbb{R}^3\)
\(A^T \vec x\), where \(\vec x \in \mathbb{R}^5\)
\(\vec x^T A^T A \vec x\), where \(\vec x \in \mathbb{R}^3\)
b)
(3 pts) Evaluate \(A\begin{bmatrix} 3 \\ 0 \\ -2 \end{bmatrix}\).
There are two interpretations of the resulting vector, based on what we’ve seen in Chapter 5.1 — what are they?
c)
(5 pts) In both subparts, try and find a vector \(\vec x \in \mathbb{R}^3\) such that \(A \vec x = \vec b\). If it’s not possible to do so, explain why.
\(\vec b = \begin{bmatrix} 0 \\ 5 \\ 3 \\ -1 \\ 4 \end{bmatrix}\)
\(\vec b = \begin{bmatrix} 10 \\ 1 \\ -17 \\ -14 \\ -4 \end{bmatrix}\)
d)
(3 pts) Explain why it’s the case that — for this particular matrix \(A\) — if \(A \vec x_1 = \vec b\) and \(A \vec x_2 = \vec b\), then \(\vec x_1 = \vec x_2\).
Problem 4: Correlation, Revisited (10 pts)
In this problem, we’ll see how the correlation coefficient between two variables, \(r\), can be expressed as a matrix multiplication.
Consider a dataset of \(n\) points, \((x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)\), and let
$$ D = \begin{bmatrix} x_1 - \bar{x} & y_1 - \bar{y} \\\\ x_2 - \bar{x} & y_2 - \bar{y} \\\\ \vdots & \vdots \\\\ x_n - \bar{x} & y_n - \bar{y} \end{bmatrix} $$
where \(\bar{x}\) and \(\bar{y}\) are the means of \(x\) and \(y\), respectively. Note that \(D\) is an \(n \times 2\) matrix, and it is mean-centered, meaning that the mean of each column is 0.
Define the matrix \(\Sigma\) as follows.
$$ \Sigma = \frac{1}{n} D^TD $$
\(\Sigma\) is a \(2 \times 2\) matrix. Its name is pronounced “sigma”, just like in summation notation and standard deviation. Don’t confuse it with summation notation; \(\Sigma\) is just a single matrix.
a)
(4 pts) For this particular matrix \(D\), find \(\Sigma\). All four components of \(\Sigma\) should be expressions involving the points \(x_1, x_2, \ldots, x_n\) and/or \(y_1, y_2, \ldots, y_n\). Feel free to use summation notation in your answers.
b)
(2 pts) In English, what do the two elements on the diagonal (top-left and bottom-right) of \(\Sigma\) represent?
c)
(2 pts) You should notice that \(\Sigma\) is a symmetric matrix, meaning \(\Sigma^T = \Sigma\). (See Chapter 5.2 for more on symmetric matrices.) The elements off the diagonal (top-right and bottom-left) are both equal, and are called the covariance of \(x\) and \(y\). For that reason, \(\Sigma\) is often called the covariance matrix.
Find an expression for the off-diagonal elements of \(\Sigma\) in terms of the correlation coefficient, \(r\), \(\sigma_x\), and \(\sigma_y\), but with no summation notation or other variables.
Hint: This only requires 1-2 lines of work. Remember the definition of \(r\) from Chapter 2.4.
d)
(2 pts) In general, suppose \(X \in \mathbb{R}^{n \times d}\) is a matrix containing \(n\) observations for each of \(d\) variables/features. The covariance matrix of \(X\) is defined similarly.
$$ \Sigma = \frac{1}{n} X^TX $$
In English, explain what the element in row 3 and column 5 of this \(\Sigma\) represents.
Problem 5: Projections, Revisited (20 pts)
As we first saw in Chapter 3.4, the projection of \(\vec u\) onto \(\vec v\) is the vector \(\displaystyle \vec p = \left( \frac{\vec u \cdot \vec v}{\vec v \cdot \vec v} \right) \vec v\).
If we assume that \(\vec v\) is a unit vector, meaning \(\lVert \vec v \rVert = 1\), then the projection of \(\vec u\) onto \(\vec v\) has a simpler form,
$$ \vec p = (\vec u \cdot \vec v) \vec v $$
Assume that \(\vec u = \begin{bmatrix} u_1 \\ u_2 \end{bmatrix}\) is some arbitrary (not-necessarily unit) vector in \(\mathbb{R}^2\), and \(\vec v = \begin{bmatrix} v_1 \\ v_2 \end{bmatrix}\) is a unit vector in \(\mathbb{R}^2\).
a)
(4 pts) Find a \(2 \times 2\) matrix \(P\), called a projection matrix, such that
$$ P \vec u = \vec p = (\vec u \cdot \vec v) \vec v $$
Think of \(P\) as a matrix that transforms \(\vec u\) into an approximation of it, in the direction of \(\vec v\) (or “projects” \(\vec u\) onto \(\vec v\)).
Hint: Start by writing \(P = \begin{bmatrix} a & b \\ c & d \end{bmatrix}\) and solve for \(a, b, c, d\) in terms of \(v_1\) and \(v_2\); \(P\) should not involve \(u_1\) or \(u_2\). Don’t forget that \(\vec v\) is a unit vector, and both \(\vec u, \vec v \in \mathbb{R}^2\).
b)
(4 pts) Find the projection of \(\vec u = \begin{bmatrix} 9 \\ -3 \end{bmatrix}\) onto the unit vector \(\vec v = \begin{bmatrix} 3 / 5 \\ 4 / 5 \end{bmatrix}\) using:
The formula for the projection of \(\vec u\) onto \(\vec v\)
The projection matrix \(P\) you found in part a)
c)
(4 pts) Show that \(P\) satisfies the following property:
$$ P^2 = P $$
This means that \(P\) is an idempotent matrix, meaning that applying \(P\) twice (or three times, or four times, etc.) to a vector is the same as applying it once.
Hint: You’ll likely end up with terms of the form \(v_1^4\). Remember that \(\vec v\) is a unit vector; use this to help you simplify.
d)
(1 pt) Use the supplemental Jupyter Notebook we’ve created for Homework 5, which can either be found here on DataHub, or here in the course GitHub repository.
Go to Problem 5d and complete the tasks in that section. There is no autograded code for Problem 5d – instead, include screenshots of your code and its output when you drag the slider to \(30^\circ\).
e)
(2 pts) Reflect on what you observed above. What angle does the dashed segment make with the projection line? Try a horizontal line, a vertical line, and a tilted line using the slider. Explain how what you see illustrates \(P^2=P\).
f)
(5 pts) Find a matrix \(R\) such that \(R\vec u\) is the reflection of \(\vec u\) across the line spanned by the unit vector \(\vec v\), using the reflection operation you learned about in Homework 4, Problem 4. Express \(R\) in terms of \(P\) and the identity matrix \(I\).
Then show algebraically that \(R^2=I\). Use your equation for \(R\) and the identity \(P^2=P\), rather than expanding the individual entries of \(R\).
Problem 6: Orthogonal Matrices (20 pts)
Read the Orthogonal Matrices section of Chapter 5.2 before starting this problem.
a)
(6 pts) Which of the following matrices are orthogonal? For each matrix, explain why or why not it is an orthogonal matrix.
$$ A=\begin{bmatrix}3&0\\\\0&2\\\\4&0\end{bmatrix},\qquad B=\begin{bmatrix}0&0&1\\\\3/5&4/5&0\\\\4/5&-3/5&0\end{bmatrix},\qquad C=\begin{bmatrix}3/5&0&1/\sqrt2\\\\4/5&0&1/\sqrt2\\\\0&1&0\end{bmatrix} $$
b)
(3 pts) Explain why the following statement is true: If \(Q\) is an orthogonal matrix, then the rows of \(Q\) form an orthonormal set, in addition to the columns. Hint: Think about what \(Q^TQ\) and \(QQ^T\) each are.
c)
(2 pts) Use the supplemental Jupyter Notebook we’ve created for Homework 5, which can either be found here on DataHub, or here in the course GitHub repository.
Go to Problem 6c and complete the tasks in that section. There is no autograded code for this problem. Instead, include screenshots of your code and its output when you drag the slider to \(30^\circ\), along with your written observations.
d)
(3 pts) Prove that an orthogonal matrix preserves the norm of every vector: if \(Q\in\mathbb R^{n\times n}\) is orthogonal and \(\vec x\in\mathbb R^n\), then \(\lVert Q\vec x\rVert=\lVert\vec x\rVert\).
e)
(4 pts) Where does the rotation matrix come from? Suppose \(\vec u\) and \(\vec v\) are unit vectors in \(\mathbb R^2\). The angle from the positive \(x_1\)-axis to \(\vec u\) is \(\alpha\), and rotating \(\vec u\) counterclockwise by \(\theta\) gives \(\vec v\).
Write \(\vec u\) and \(\vec v\) in terms of \(\alpha\) and \(\theta\).
Use the identities below to find a \(2\times2\) matrix \(R\) such that \(R\vec u=\vec v\). Your matrix should involve \(\theta\), but not \(\alpha\).
Explain why the same matrix rotates vectors that aren’t unit vectors by \(\theta\), too.
Hint: Use the angle-addition identities
$$ \cos(\alpha+\theta)=\cos\alpha\cos\theta-\sin\alpha\sin\theta,\qquad \sin(\alpha+\theta)=\sin\alpha\cos\theta+\cos\alpha\sin\theta $$
f)
(2 pts) Prove that the rotation matrix \(R\) you found in part e) is orthogonal for every angle \(\theta\).
Hint: Use \(\cos^2(\theta)+\sin^2(\theta)=1\).
Problem 7: Rank and CR Decomposition (10 pts)
The rank of a matrix is the dimension of the span of its columns. We call the span of a matrix’s columns its column space.
Read the CR Decomposition section of Chapter 5.4 before starting this problem. There, we introduce the concept of a CR decomposition. As another example, let \(A\) be the matrix from Homework 4, Problem 6.
Its CR decomposition is:
$$ A = \underbrace{\begin{bmatrix} 5 & 3 & 2 \\\\ 3 & 0 & 4 \\\\ -2 & 0 & 3 \\\\ 8 & 2 & -8 \\\\ 1 & 1 & 0 \end{bmatrix}}_{C} \underbrace{\begin{bmatrix} 1 & 0 & -2 & 0 \\\\ 0 & 1 & 5 & 0 \\\\ 0 & 0 & 0 & 1 \end{bmatrix}}_{R} $$
To understand where the numbers in \(R\) came from, read the solutions to Homework 4, linked above.
For each matrix below, by hand:
Find a CR decomposition, choosing the columns of \(C\) from left to right.
Find the rank of \(A\). How is it related to the number of columns in \(C\)?
Can you find a nonzero vector \(\vec v\) such that \(A\vec v=\vec 0\)? If so, give an example. If not, explain why not.
Hint: Look for relationships between the columns; this is not meant to be time consuming.
a)
(3 pts) \(A=\begin{bmatrix}1&2&3\\4&5&6\\7&8&9\end{bmatrix}\)
b)
(3 pts) \(A=\begin{bmatrix}3&5\\1&1\\2&-4\\30&0\end{bmatrix}\)
c)
(4 pts) \(A=\begin{bmatrix}1&-2&3&-1\\-2&4&1&-5\\3&-6&4&2\\0&0&5&-5\end{bmatrix}\)
Problem 8: Random Matrices, Rank, and Cosine Similarity (8 pts)
Use the supplemental Jupyter Notebook we’ve created for Homework 5, which can either be found here on DataHub, or here in the course GitHub repository.
Go to Problem 8. For this problem, you will use simulations to explore the typical rank of a randomly generated matrix, and see how the cosine similarities between randomly generated vectors change as their number of components increases.
All code is provided. There is no autograded code or notebook submission for this problem. Include the following in your regular Homework 5 PDF under Problem 8:
Task 1 (1 pt): Your handwritten explanation of the Gaussian matrix ranks.
Task 2 (2 pts): Your handwritten explanation of the Gaussian matrix product ranks.
Task 3 (3 pts): Your handwritten responses to both questions about binary matrix product ranks.
Task 4 (2 pts): Your handwritten responses to both questions about cosine similarity.
Your submission time for Homework 5 is the later of the two submission times: your Problem 1 video link and your regular Homework 5 PDF.