Lab 6: Rank, Column Space, Null Space, and Inverses

due for completion at 11:59PM Ann Arbor Time on Wednesday, May 27th, 2026

Each lab worksheet will contain several activities, some of which will involve writing code and others that will involve writing math on paper. To receive credit for a lab, you must complete as many of the activities as you can in 2 hours and submit a PDF of your work to Gradescope. We will provide specific instructions on how to submit programming activities (e.g. submitting the notebook or including a screenshot of some output).

Feel free to work with others in the course, but you must submit individually.


Activities


Note: You may find it helpful to work on the first few problems of Homework 5 before starting this lab.

Recap: Rank, Column Space, and Null Space (Chapter 5.3 and 5.4)

Suppose \(A\) is an \(n \times d\) matrix. Then, \(\text{rank}(A)\) is the number of linearly independent columns of \(A\).

 NotationDescriptionDimensionSubspace of
Column space\(\text{colsp}(A)\)Span of the columns of \(A\)\(\text{rank}(A)\)\(\mathbb{R}^n\)
Row space\(\text{colsp}(A^T)\)Span of the rows of \(A\)\(\text{rank}(A)\)\(\mathbb{R}^d\)
Null space\(\text{nullsp}(A)\)Set of all vectors \(\vec{x}\) such that \(A\vec{x} = \vec{0}\)\(d - \text{rank}(A)\)\(\mathbb{R}^d\)

Additionally, note that you can write the dot product of two vectors \(\vec u, \vec v \in \mathbb{R}^n\) as \(\vec u^T\vec v\):

\(\vec u^T = \begin{bmatrix}u_1 & u_2 & \cdots & u_n\end{bmatrix} \qquad \vec v = \begin{bmatrix}v_1 \\ \vdots \\ v_n\end{bmatrix}\)

\(\displaystyle \vec u^T\vec v = u_1v_1 + \dots + u_nv_n = \sum_{i=1}^{n}(u_iv_i) = \vec u \cdot \vec v\) (not \(\vec u^T \cdot \vec v\))


Activity 1: Null Space of a Matrix with Linearly Independent Columns

Let \(A = \begin{bmatrix} 3 & 0 \\ 0 & 4 \\ 1 & 0 \end{bmatrix}\). What is \(\text{nullsp}(A)\)?

Solution

We are asked to find the null space of

$$ A = \begin{bmatrix} 3 & 0 \\\\ 0 & 4 \\\\ 1 & 0 \end{bmatrix} $$

The null space of \(A\), denoted \(\text{nullsp}(A)\), is the set of all vectors \(\vec{x}\) such that

$$ A\vec{x} = \vec{0} $$

In practice, we know that since \(A\)’s columns are all linearly independent, we know that the only vector in its nullspace is \(\vec 0 = \begin{bmatrix} 0 \\ 0 \end{bmatrix}\). This is because the only linear combination of linearly independent vectors that gives the zero vector is the trivial combination, where all coefficients are zero. So,

$$ \text{nullsp}(A) = \{ \vec 0 \} = \{ \begin{bmatrix} 0 \\\\ 0 \end{bmatrix} \} $$

But, let’s suppose we didn’t realize this, and wanted to systematically solve the system of equations to find the null space. Let \(\vec{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}\) Then:

$$ A\vec{x} = \begin{bmatrix} 3 & 0 \\\\ 0 & 4 \\\\ 1 & 0 \end{bmatrix} \begin{bmatrix} x_1 \\\\ x_2 \end{bmatrix} = \begin{bmatrix} 3x_1 \\\\ 4x_2 \\\\ x_1 \end{bmatrix} $$
$$ \begin{bmatrix} 3x_1 \\\\ 4x_2 \\\\ x_1 \end{bmatrix} = \begin{bmatrix} 0 \\\\ 0 \\\\ 0 \end{bmatrix} $$

This gives the system of equations:

$$ \begin{align*} 3x_1 &= 0 \\\\ 4x_2 &= 0 \\\\ x_1 &= 0 \end{align*} $$
$$ x_1 = 0 \quad x_2 = 0 $$

The only vector that satisfies this system is

$$ \vec{x} = \begin{bmatrix} 0 \\\\ 0 \end{bmatrix} $$

which confirms our initial approach.


Activity 2: Fundamentals

Let \(X=\begin{bmatrix}1 & 2 & -1 & 3 & 4 & 4 \\ 2 & 5 & -2 & 7 & 11 & 10 \\ 4 & 8 & -4 & 12 & 16 & 16\end{bmatrix}\).

a)

Find a basis for \(\text{colsp}(X)\). What is \(\text{rank}(X)\)? Why? Hint: Column 5 is a linear combination of columns 1 and 2. With this fact, you should be able to answer this relatively quickly.

Solution

We are given

$$ X = \begin{bmatrix} 1 & 2 & -1 & 3 & 4 & 4 \\\\ 2 & 5 & -2 & 7 & 11 & 10 \\\\ 4 & 8 & -4 & 12 & 16 & 16 \end{bmatrix} $$

Let the columns of \(X\) be

$$ \vec{v}_1 = \begin{bmatrix}1\\\\2\\\\4\end{bmatrix} \quad \vec{v}_2 = \begin{bmatrix}2\\\\5\\\\8\end{bmatrix} \quad \vec{v}_3 = \begin{bmatrix}-1\\\\-2\\\\-4\end{bmatrix} \quad \vec{v}_4 = \begin{bmatrix}3\\\\7\\\\12\end{bmatrix} \quad \vec{v}_5 = \begin{bmatrix}4\\\\11\\\\16\end{bmatrix} \quad \vec{v}_6 = \begin{bmatrix}4\\\\10\\\\16\end{bmatrix} $$

The “textbook” way to find a basis for \(\text{colsp}(X)\) is to use the algorithm from Chapter 4.2 to find a linearly independent subset of \(S\) that spans it (where \(S = \text{colsp}(X)\)).

Given \(\vec{v}_1, \vec{v}_2, \dots, \vec{v}_d\)

Initialize \(S = \lbrace\vec{v}_1\rbrace\)

For \(i = 2, \dots, d\)

If \(\vec{v}_i\) is not a linear combination of \(S\), add \(\vec{v}_i\) to \(S\)

But, we’ve picked the numbers such that the relationships are relatively easy to see. Reading from left to right:

  1. \(\vec v_1\) is the first vector we’ve seen, so it’s linearly independent and we add it to our basis.

  2. \(\vec v_2\) is not a multiple of \(\vec v_1\), so we add it to our basis.

  3. \(\vec v_3 =- \vec v_1\), so don’t add it.

  4. \(\vec v_4 = \vec v_1 + \vec v_2\), so don’t add it.

  5. \(\vec v_5 = -2 \vec v_1 + 3 \vec v_2\), so don’t add it.

  6. \(\vec v_6 = 2 \vec v_2\), so don’t add it.

So,

$$ \boxed{ \text{Basis for } \text{colsp}(X) = S = \left\{ \begin{bmatrix}1\\\\2\\\\4\end{bmatrix}, \begin{bmatrix}2\\\\5\\\\8\end{bmatrix} \right\} \quad \text{rank}(X) = 2 } $$

The rank equals the number of linearly independent columns identified by the algorithm.

b)

Fill in the blanks: \(\text{colsp}(X^T)\) is a ____-dimensional subspace of ____.

Solution

We know that \(\text{colsp}(X^T)\) is the row space of \(X\) and from the previous part \(\text{rank}(X) = 2\).

The dimension of the row space is equal to the rank of the matrix:

$$ \dim(\text{colsp}(X^T)) = \text{rank}(X) = 2 $$

The rows of \(X\) each lie in \(\mathbb{R}^6\) so the row space is a subspace of \(\mathbb{R}^6\).

$$ \boxed{ \text{colsp}(X^T) \text{ is a 2-dimensional subspace of } \mathbb{R}^6 } $$
c)

Fill in the blanks: \(\text{nullsp}(X)\) is a ____-dimensional subspace of ____.

Solution

From Chapter 5.4, the rank-nullity theorem states that for any \(n \times d\) matrix \(X\),

$$ \text{rank}(X) + \dim(\text{nullsp}(X)) = \underbrace{d}_{\text{\# columns in } X} $$

The rank of \(X\) was found to be \(2\), and \(X\) has \(6\) columns, so

$$ 2 + \dim(\text{nullsp}(X)) = 6 $$
$$ \dim(\text{nullsp}(X)) = 6 - 2 = 4 $$

By definition, \(\text{nullsp}(X)\) is the set of all vectors \(\vec{v} \in \mathbb{R}^6\) such that \(X\vec{v} = \vec{0}\). Therefore, it is a subspace of \(\mathbb{R}^6\).

$$ \boxed{ \text{nullsp}(X) \text{ is a 4-dimensional subspace of } \mathbb{R}^6 } $$
d)

Find a basis for \(\text{nullsp}(X)\). Hint: You should be able to answer this without solving equations.

Solution

The “easy way” to find a basis for \(\text{nullsp}(X)\) is to find all of the ways that \(\vec 0\) can be written as a linear combination of \(X\)’s columns. The previous part told us that \(\text{dim}(\text{nullsp}(X)) = 4\), so we need to find 4 linearly independent vectors that satisfy \(X\vec{v} = \vec{0}\).

Recall from the solutions to part a) that \(\lbrace\vec v_1, \vec v_2\rbrace\) form a basis for \(\text{colsp}(X)\), meaning the other four columns of \(X\) can be written as linear combinations of them. Rearranging these linear independence relationships gives us ways to write \(\vec 0\):

  • \(\vec v_3 =- \vec v_1 \implies \vec v_1 + \vec v_3 = \vec 0\)

  • \(\vec v_4 = \vec v_1 + \vec v_2 \implies \vec v_1 + \vec v_2 - \vec v_4 = \vec 0\)

  • \(\vec v_5 = -2 \vec v_1 + 3 \vec v_2 \implies -2 \vec v_1 + 3 \vec v_2 - \vec v_5 = \vec 0\)

  • \(\vec v_6 = 2 \vec v_2 \implies 2 \vec v_2 - \vec v_6 = \vec 0\)

Let’s look at the first bullet point. \(\vec v_1 + \vec v_3 = \vec 0\) implies that:

$$ X \begin{bmatrix} 1 \\\\ 0 \\\\ 1 \\\\ 0 \\\\ 0 \\\\ 0 \end{bmatrix} = \begin{bmatrix} 1 & 2 & -1 & 3 & 4 & 4 \\\\ 2 & 5 & -2 & 7 & 11 & 10 \\\\ 4 & 8 & -4 & 12 & 16 & 16 \end{bmatrix} \begin{bmatrix} 1 \\\\ 0 \\\\ 1 \\\\ 0 \\\\ 0 \\\\ 0 \end{bmatrix} = \vec 0 $$

The second bullet point, \(\vec v_1 + \vec v_2 - \vec v_4 = \vec 0\) implies that:

$$ X \begin{bmatrix} 1 \\\\ 1 \\\\ 0 \\\\ -1 \\\\ 0 \\\\ 0 \end{bmatrix} = \vec 0 $$

and similarly with \(\begin{bmatrix} -2 \\ 3 \\ 0 \\ 0 \\ -1 \\ 0 \end{bmatrix}\) (bullet point 3, i.e. from \(-2 \vec v_1 + 3 \vec v_2 - \vec v_5 = \vec 0\)) and \(\begin{bmatrix} 0 \\ 2 \\ 0 \\ 0 \\ 0 \\ -1 \end{bmatrix}\) (bullet point 4, i.e. from \(2 \vec v_2 - \vec v_6 = \vec 0\)).

So, a basis for \(\text{nullsp}(X)\) is:

$$ \boxed{ \left\{ \begin{bmatrix} 1 \\\\ 0 \\\\ 1 \\\\ 0 \\\\ 0 \\\\ 0 \end{bmatrix}, \begin{bmatrix} 1 \\\\ 1 \\\\ 0 \\\\ -1 \\\\ 0 \\\\ 0 \end{bmatrix}, \begin{bmatrix} -2 \\\\ 3 \\\\ 0 \\\\ 0 \\\\ -1 \\\\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\\\ 2 \\\\ 0 \\\\ 0 \\\\ 0 \\\\ -1 \end{bmatrix} \right\} } $$

Notice that these vectors are linearly independent, because each successive vector has a non-zero entry in a position that the previous vectors all have zeros in. They span the entirety of \(\text{nullsp}(X)\) — any linear combination of them will, when multiplied by \(X\), give the zero vector.

Suppose \(A\) is an \(n \times d\) matrix with rank \(r\). A CR decomposition of \(A\) is a product of two matrices \(C\) and \(R\), where \(A = CR\) and:

  • \(C\) is an \(n \times r\) matrix and \(R\) is a \(r \times d\) matrix

  • \(C\) contains the linearly independent columns of \(A\), selected left-to-right

  • \(R\) tells how to “mix’’ the columns of \(C\) (which are linearly independent) to reconstruct the columns of \(A\)

Let’s keep working with \(X = \begin{bmatrix}1 & 2 & -1 & 3 & 4 & 4 \\ 2 & 5 & -2 & 7 & 11 & 10 \\ 4 & 8 & -4 & 12 & 16 & 16\end{bmatrix}\).

e)

Find a CR decomposition of \(X\). This shouldn’t take very much work; review your work from part a) in finding a basis for \(\text{colsp}(X)\).

Solution

We are asked to find a CR decomposition of

$$ X = \begin{bmatrix} 1 & 2 & -1 & 3 & 4 & 4 \\\\ 2 & 5 & -2 & 7 & 11 & 10 \\\\ 4 & 8 & -4 & 12 & 16 & 16 \end{bmatrix} $$

From part a), we determined that

$$ \text{rank}(X) = 2 \quad \text{and} \quad \text{basis for } \text{colsp}(X) = \left\{ \begin{bmatrix} 1 \\\\ 2 \\\\ 4 \end{bmatrix}, \begin{bmatrix} 2 \\\\ 5 \\\\ 8 \end{bmatrix} \right\} $$

These two columns of \(X\) are linearly independent, and all other columns of \(X\) can be written as linear combinations of them.

By definition of the CR decomposition, the matrix \(C\) contains the linearly independent columns of \(X\) (selected left-to-right). Thus,

$$ C = \begin{bmatrix} 1 & 2 \\\\ 2 & 5 \\\\ 4 & 8 \end{bmatrix} $$

\(C\) has shape \(3\times 2\), and its two columns form a basis for \(\text{colsp}(X)\).

The matrix \(R\) describes how to “mix’’ the columns of \(C\) to obtain all six columns of \(X\). That is, each column of \(R\) contains the coefficients expressing one column of \(X\) as a linear combination of the two columns of \(C\).

Fortunately, we did all of the heavy lifting in part d), when finding a basis for \(\text{nullsp}(X)\). We just need to rearrange our work. Recall that:

$$ \vec v_3 = -\vec v_1, \quad \vec v_4 = \vec v_1 + \vec v_2, \quad \vec v_5 = -2 \vec v_1 + 3 \vec v_2, \quad \vec v_6 = 2 \vec v_2 $$

So,

$$ R = \begin{bmatrix} 1 & 0 & -1 & 1 & -2 & 0\\\\ 0 & 1 & 0 & 1 & 3 & 2 \end{bmatrix} $$

Where did the first two columns of \(R\) come from? We want the first column of \(X\) to be just the first column of \(C\), which we get by taking 1 of column 1 and 0 of column 2. Similarly, the second column of \(X\) is just the second column of \(C\), which we get by taking 0 of column 1 and 1 of column 2.

So, in a CR decomposition that is constructed by taking the linearly independent columns of \(X\) from left-to-right, the first \(r\) columns are the \(r \times r\) identity matrix, and the remaining columns are the coefficients needed to write the remaining \(d - r\) columns of \(X\) as linear combinations of the \(r\) columns of \(C\).

Verifying,

$$ C R = \begin{bmatrix} 1 & 2 \\\\ 2 & 5 \\\\ 4 & 8 \end{bmatrix} \begin{bmatrix} 1 & 0 & -1 & 1 & -2 & 0\\\\ 0 & 1 & 0 & 1 & 3 & 2 \end{bmatrix} = \begin{bmatrix} 1 & 2 & -1 & 3 & 4 & 4\\\\ 2 & 5 & -2 & 7 & 11 & 10\\\\ 4 & 8 & -4 & 12 & 16 & 16 \end{bmatrix} = X $$
$$ \boxed{ C = \begin{bmatrix} 1 & 2 \\\\ 2 & 5 \\\\ 4 & 8 \end{bmatrix} \quad R = \begin{bmatrix} 1 & 0 & -1 & 1 & -2 & 0\\\\ 0 & 1 & 0 & 1 & 3 & 2 \end{bmatrix} \quad \text{and } X = C R } $$

The key idea being assessed here is that in \(A = CR\), the columns of \(C\) are linearly independent and a basis for \(\text{colsp}(A)\), while the rows of \(R\) are linearly independent and a basis for \(\text{colsp}(A^T)\)!


Activity 3: Outer Products

Suppose \(A = \vec u \vec v^T + \vec w \vec z^T\), where \(\vec u, \vec v, \vec w, \vec z \in \mathbb{R}^n\) are non-zero vectors.

a)

What is \(\text{rank}(\vec u \vec v^T)\)?

Solution

The matrix \(\vec{u}\vec{v}^T\) is formed by multiplying a column vector \(\vec{u}\) by a row vector \(\vec{v}^T\):

$$ \vec{u}\vec{v}^T = \begin{bmatrix} u_1 \\\\[2pt] u_2 \\\\[2pt] \vdots \\\\[2pt] u_n \end{bmatrix} \begin{bmatrix} v_1 & v_2 & \cdots & v_n \end{bmatrix} = \begin{bmatrix} u_1v_1 & u_1v_2 & \cdots & u_1v_n \\\\[2pt] u_2v_1 & u_2v_2 & \cdots & u_2v_n \\\\[2pt] \vdots & \vdots & \ddots & \vdots \\\\[2pt] u_nv_1 & u_nv_2 & \cdots & u_nv_n \end{bmatrix} $$

Each column of this matrix is a scalar multiple of the same vector \(\vec{u}\). For example:

$$ \text{First column: } v_1\vec{u}, \quad \text{Second column: } v_2\vec{u}, \quad \dots, \quad \text{n-th column: } v_n\vec{u}. $$

Thus, every column lies in the same one-dimensional subspace spanned by \(\vec{u}\):

$$ \text{colsp}(\vec{u}\vec{v}^T) = \text{span}(\{\vec{u}\}) $$

Since all columns are proportional to \(\vec{u}\), there is exactly one linearly independent column. The dimension of the column space (and hence the rank) is therefore 1:

$$ \boxed{\text{rank}(\vec{u}\vec{v}^T) = 1} $$
b)

Under what conditions is \(\text{rank}(A) = 2\)? What about \(\text{rank}(A) < 2\)? Hint: First, think about what happens when multiplying \(A\) by a vector \(\vec x \in \mathbb{R}^n\). Can you write this as a linear combination of some other vectors? The case for \(\text{rank}(A) = 2\) is more complicated than “\(\vec u\) and \(\vec w\) are linearly independent.”

Solution

The short answer is that \(\text{rank}(A) = 2\) if and only if \(\vec u\) and \(\vec w\) are linearly independent and \(\vec v\) and \(\vec z\) are linearly independent. If either \(\lbrace \vec u, \vec w \rbrace\) or \(\lbrace \vec v, \vec z \rbrace\) are linearly dependent, then \(\text{rank}(A) = 1\). The easy way to reason about this is that:

  • The columns of \(\vec u \vec v^T\) are all scalar multiples of \(\vec u\) and the columns of \(\vec w \vec z^T\) are all scalar multiples of \(\vec w\). If \(\vec u\) and \(\vec w\) are linearly dependent, then all columns of \(A\) are scalar multiples of \(\vec u\) (or \(\vec w\)), meaning \(\text{dim}(\text{colsp}(A)) = 1\), which means \(\text{rank}(A) = 1\).

  • The rows of \(\vec u \vec v^T\) are all scalar multiples of \(\vec v^T\) and the rows of \(\vec w \vec z^T\) are all scalar multiples of \(\vec z^T\). If \(\vec v\) and \(\vec z\) are linearly dependent, then all rows of \(A\) are scalar multiples of \(\vec v^T\) (or \(\vec z^T\)), meaning \(\text{dim}(\text{rowsp}(A)) = 1\), which means \(\text{rank}(A) = 1\).

  • In order for \(\text{rank}(A) = 2\), there need to be at least two linearly independent columns and two linearly independent rows, which means that both pairs \(\lbrace \vec u, \vec v \rbrace\) and \(\lbrace \vec w, \vec z \rbrace\) must be linearly independent.

For a more detailed explanation, let’s start from the beginning.

We are given

$$ A = \vec{u}\vec{v}^T + \vec{w}\vec{z}^T $$

with each outer product being rank 1. As the hint suggests, let’s think about what happens when multiplying \(A\) by a vector \(\vec x \in \mathbb{R}^n\):

$$ A \vec x = (\vec{u}\vec{v}^T + \vec{w}\vec{z}^T) \vec x = (\vec{v}^T\vec{x})\vec{u} + (\vec{z}^T\vec{x})\vec{w} $$

This shows that \(A \vec x\) is always a linear combination of \(\vec{u}\) and \(\vec{w}\). So, \(\text{colsp}(A)\) is a subspace of \(\text{span}(\lbrace\vec{u}, \vec{w}\rbrace)\), since every vector in \(\text{colsp}(A)\) can be written as a linear combination of \(\vec{u}\) and \(\vec{w}\).

  • If \(\vec{u}\) and \(\vec{w}\) themselves point in the same direction (i.e. are linearly dependent), then any vector in \(\text{colsp}(A)\) is a scalar multiple of \(\vec u\) (or, equivalently, of \(\vec w\)). This would mean \(\text{dim}(\text{colsp}(A)) = 1\), which means \(\text{rank}(A) = 1\).

  • With that case out of the way, let’s suppose \(\vec u\) and \(\vec w\) are linearly independent. Does this automatically mean that \(\text{dim}(\text{colsp}(A)) = 2\)? No: all we know for sure is that every vector of the form \(A \vec x\) is a linear combination of \(\vec u\) and \(\vec w\). We don’t yet know that the converse is true, i.e. that every linear combination of \(\vec u\) and \(\vec w\) can be written as \(A \vec x\), for some \(\vec x \in \mathbb{R}^n\).

In order for it to be the case that \(\text{colsp}(A) = \text{span}(\lbrace\vec{u}, \vec{w}\rbrace)\), we must be able to take any linear combination of \(\vec{u}\) and \(\vec{w}\) and write it as \(A \vec x\) for some \(\vec x \in \mathbb{R}^n\). Meaning, for any \(c\) and \(d\), we must be able to find some \(\vec x\) such that:

$$ A \vec x = c \vec u + d \vec w $$

But, we know that

$$ A \vec x = (\vec{v}^T\vec{x})\vec{u} + (\vec{z}^T\vec{x})\vec{w} $$

So, the question really is, is it always possible, for any \(c\), \(d\), to find an \(\vec x\) that satisfies:

$$ c = \vec{v}^T\vec{x}, \qquad d = \vec{z}^T\vec{x} $$

This may seem a bit abstract, so let’s just plug in specific numbers for \(c\) and \(d\) to make things a bit more clear.

$$ 3 = \vec{v}^T\vec{x}, \qquad 4 = \vec{z}^T\vec{x} $$

Remember that \(\vec v\) and \(\vec z\) are fixed here. Given them, can we find an \(\vec x\) that satisfies both of these equations? We can, as long as \(\vec v\) and \(\vec z\) are linearly independent too! If they are, then \(\vec v^T \vec x = 3\) is one equation with \(n\) unknowns, and \(\vec z^T \vec x = 4\) is another equation with \(n\) unknowns. Since there are \(n\) unknowns and 2 equations, the system is overdetermined, and there are infinitely many solutions for \(\vec x\), we just need to pick one of them.

Again, let’s give an example. Suppose \(\vec u\) and \(\vec w\) are linearly independent (a prerequisite for the case we’re considering), let \(\vec v = \begin{bmatrix} 3 \\ 4 \\ 5 \end{bmatrix}\) and \(\vec z = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}\) (which are also linearly independent as a pair), and suppose \(A \vec x = 3 \vec u + 4 \vec w\). Then, \(\vec x = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}\) must satisfy:

$$ \vec v^T \vec x = 3 \implies 3x_1 + 4x_2 + 5x_3 = 3 $$
$$ \vec z^T \vec x = 4 \implies 0x_1 + 0x_2 + 1x_3 = 4 $$

Solving these, we get:

$$ x_3 = 4 $$
$$ 3x_1 + 4x_2 + 20 = 3 \implies 3x_1 + 4x_2 = -17 $$

So, as long as we pick \(x_3 = 4\) and \(x_1\) and \(x_2\) to satisfy \(3x_1 + 4x_2 = -17\), we can find an \(\vec x\) that satisfies both of the equations, meaning we’re able to find an \(\vec x\) to make \(A \vec x = 3 \vec u + 4 \vec w\).

Nothing was special about \(c = 3\) and \(d = 4\). What was special was that in addition to \(\lbrace \vec u, \vec w \rbrace\) being linearly independent, we also had \(\lbrace \vec v, \vec z \rbrace\) being linearly independent. This is the condition that makes it possible to find an \(\vec x\) that satisfies both of the equations. If they weren’t linearly independent, it isn’t guaranteed that we can find an \(\vec x\) that satisfies both of the equations. Suppose we revisit the same example, but instead with \(\vec v = \begin{bmatrix} 3 \\ 4 \\ 5 \end{bmatrix}\) and \(\vec z = \begin{bmatrix} 30 \\ 40 \\ 50 \end{bmatrix}\) (which are linearly dependent as a pair). Then, \(\vec v^T \vec x = 3\) and \(\vec z^T \vec x = 4\) are two equations with \(n\) unknowns still, but \(\vec x\) would need to satisfy

$$ 3x_1 + 4x_2 + 5x_3 = 3 $$
$$ 30x_1 + 40x_2 + 50x_3 = 4 $$

which has no solutions, meaning we’re not able to find an \(\vec x\) to make \(A \vec x = 3 \vec u + 4 \vec w\), meaning \(\text{colsp}(A) \neq \text{span}(\lbrace\vec{u}, \vec{w}\rbrace)\).

That was a long solution, in which we tried to build intuition for how you might think about this. But to summarize:

  • \(\text{rank}(A) = 2\) if and only if \(\vec u\) and \(\vec w\) are linearly independent and \(\vec v\) and \(\vec z\) are linearly independent.

  • If either \(\lbrace \vec u, \vec w \rbrace\) or \(\lbrace \vec v, \vec z \rbrace\) are linearly dependent, then \(\text{rank}(A) = 1\).

  • If any of the vectors are the zero vector, then \(\text{rank}(A) = 0\).


Activity 4: The Systems of Equations View

Let \(A\) be an \(n \times d\) matrix of rank \(r\), and suppose there exists vectors \(\vec b \in \mathbb{R}^n\) such that

$$ A \vec x = \vec b $$

does not have a solution (meaning no \(\vec x\) makes \(A \vec x = \vec b\)).

a)

What are all inequalities (\(<\) or \(\le\)) that must be true between \(n\), \(d\), and \(r\)?

Solution

If \(A\vec{x} = \vec{b}\) has no solution for some \(\vec{b}\), then \(\vec{b}\) does not lie in \(\text{colsp}(A)\). Thus, the column space of \(A\) is a proper subspace of \(\mathbb{R}^n\), meaning its dimension (the rank) is strictly less than \(n\):

$$ r < n $$

By general rank properties, rank cannot exceed either the number of rows or columns:

$$ r \le n, \quad r \le d $$

Combining these gives:

$$ \boxed{r < n \quad \text{and} \quad r \le d} $$
b)

How do you know that \(A^T\vec y= \vec 0\) has solutions other than \(\vec y=\vec 0\)?

Solution

By the rank-nullity theorem,

$$ \text{rank}(A^T) + \dim(\text{nullsp}(A^T)) = \underbrace{n}_{\text{\# columns in } A^T} $$

and since \(\text{rank}(A^T)=r<n\), it follows that \(\dim(\text{nullsp}(A^T))=n-r>0\). Therefore, \(\text{nullsp}(A^T)\) contains non-zero vectors, so there must exist some \(\vec{y} \ne \vec{0}\) such that \(A^T \vec{y} = \vec{0}\).


Activity 5: Symbolic Inverses

Given that \(A\) is an invertible \(n \times n\) matrix that satisfies \(A^4 - 3A^2 + 2A - 4I = 0\), find an expression for \(A^{-1}\) in terms of \(A\).

Solution

The goal is to find another matrix \(B\) such that \(AB = BA = I\). We can do this by isolating the identity matrix, \(I\), and then trying to write the other side of the equation as \(A\) times some other matrix.

$$ \begin{align*} A^4 - 3A^2 + 2A - 4I &= 0 \\\\ A^4 - 3A^2 + 2A &= 4I \\\\ A(A^3 - 3A + 2I) &= 4I \\\\ A \left( \frac{1}{4}(A^3 - 3A + 2I) \right) &= I \end{align*} $$

So, \(\boxed{A^{-1} = \frac{1}{4}(A^3 - 3A + 2I)}\).

We derived \(A^{-1}\) by factoring out \(A\) on the left, i.e. \(A \left( \frac{1}{4}(A^3 - 3A + 2I) \right) = I\). For \(A^{-1}\) to be the inverse of \(A\), it must also be true that \(\left( \frac{1}{4}(A^3 - 3A + 2I) \right) A = I\); this fact is not automatically true in general, since the order of multiplication matters. But here, we don’t need to do any additional work, since our factorization only involved powers of \(A\), and in (say) \(A^2 = AA\) the order of multiplication doesn’t matter.


Activity 6: Basics of Invertibility

Suppose \(A\) is an \(n \times n\) matrix. State as many of the equivalent conditions for invertibility as you can.

Solution

If \(A\) is an \(n \times n\) matrix, then \(A\) is invertible if and only if:

  • \(\text{rank}(A) = n\)

  • \(A\)’s columns are linearly independent (and hence \(\text{colsp}(A) = \mathbb{R}^n\))

  • \(A\)’s rows are linearly independent (and hence \(\text{rowsp}(A) = \text{colsp}(A^T) = \mathbb{R}^n\))

  • \(A\)’s null space is only the zero vector, i.e. \(\text{nullsp}(A) = \lbrace\vec 0\rbrace\)

  • \(\text{det}(A) \neq 0\)


Activity 7: Programming

Complete the tasks in the lab06.ipynb notebook. Watch this video first with tips on using numpy for linear algebra.

There are two ways to access the supplemental Jupyter Notebook:

  • Option 1 (preferred): Set up a Jupyter Notebook environment locally, use git to clone our course repository, and open labs/lab06/lab06.ipynb. For instructions on how to do this, see the Environment Setup page of the course website.

  • Option 2: Click here to open lab06.ipynb on DataHub. Before doing so, read the instructions on the Environment Setup page on how to use the DataHub.

Once you’re done, include a screenshot of your completed Activity 7 implementation in your PDF submission of Lab 6 to Gradescope, making sure to include proof that the (local) autograder passed. Instructions on how to do this are in the lab notebook.