Linear algebra (Osnabrück 2024-2025)/Part II/Lecture 55

Example

Let ${}K$ a field. We consider two finite index sets ${}I$ and ${}J$ , and the mapping spaces ${}\operatorname {Map} \,{\left(I,K\right)}$ and ${}\operatorname {Map} \,{\left(J,K\right)}$ , which are both finite-dimensional ${}K$ -vector spaces. Is there a relation with the mapping set

\operatorname {Map} \,{\left(I\times J,K\right)}?

For mappings ${}f\in \operatorname {Map} \,{\left(I,K\right)}$ and ${}g\in \operatorname {Map} \,{\left(J,K\right)}$ , we obtain easily a mapping on ${}I\times J$ , called ${}f\otimes g$ ( ${}f$ tensor ${}g$ ) that is defined by

{}(f\otimes g)(i,j):=f(i)\cdot g(j)\,.

For the standard vectors we have

{}e_{i}\otimes e_{j}=e_{(i,j)}\,.

Every element ${}h\in \operatorname {Map} \,{\left(I\times J,K\right)}$ can be written as a linear combination of elements of the form ${}f\otimes g$ but not every ${}h$ hat this simple form. We have

{}(a_{1}f_{1}+a_{2}f_{2})\otimes g=a_{1}(f_{1}\otimes g)+a_{2}(f_{2}\otimes g)\,,

and accordingly in the second component.

In this lecture, we introduce an important construction for vector spaces, the so-called tensor product. In the special case just considered, the tensor product yields the mapping space on the product set; that is, we have

{}\operatorname {Map} \,{\left(I,K\right)}\otimes _{K}\operatorname {Map} \,{\left(I,K\right)}\cong \operatorname {Map} \,{\left(I\times J,K\right)}\,.

Here, the properties of the constructed object are more important than the construction itself. The construction is quite abstract, and rests on the construction of residue class spaces, and following construction.

For an arbitrary set of symbols ${}S$ , and a field ${}K$ , we can construct the vector space ${}K^{(S)}$ that consists of all mappings

S\longrightarrow K

that have everywhere except for finitely many places the value ${}0$ . If ${}e_{s}$ denotes the mapping that has the value ${}1$ at the position ${}s$ , and everywhere else the value ${}0$ , then ${}K^{(S)}$ consists of all finite sums

\sum _{s\in S}a_{s}e_{s}.

Hence, the ${}e_{s}$ form a basis of this space.

This construction is a special case of the direct sum of (in general) infinitely many ${}K$ -vector spaces; we take the direct sum of the vector spaces ${}K$ with itself as often as the set ${}S$ tells us.

The tensor product of vector spaces

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ be ${}K$ -vector spaces. We recall that a multilinear mapping in another ${}K$ -vector space ${}W$ is a mapping

\psi \colon V_{1}\times \cdots \times V_{n}\longrightarrow W

that is ${}K$ -linear in every component, that is, when all the other components are fixed. We want to construct a vector space ${}U$ , together with of a multilinear mapping

\varphi \colon V_{1}\times \cdots \times V_{n}\longrightarrow U

such that for every multilinear mapping ${}\psi$ as above, there exists a linear mapping

L\colon U\longrightarrow W

fulfilling ${}\psi =L\circ \varphi$ . By this construction, multilinear mappings are transformed to linear mappings on a new vector space.

Definition

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ be ${}K$ -vector spaces. Let ${}F$ be the ${}K$ -vector space generated by all symbols ${}(v_{1},\ldots ,v_{n})$ (with ${}v_{i}\in V_{i}$ , we write the basis elements as $e_{(v_{1},\ldots ,v_{n})}$ ). Let ${}U\subseteq F$ be the ${}K$ -linear subspace of ${}F$ generated by all elements of the form

${}re_{(v_{1},\ldots ,v_{i-1},v_{i},v_{i+1},\ldots ,v_{n})}-e_{(v_{1},\ldots ,v_{i-1},rv_{i},v_{i+1},\ldots ,v_{n})}$ ,
${}e_{(v_{1},\ldots ,v_{i-1},u+w,v_{i+1},\ldots ,v_{n})}-e_{(v_{1},\ldots ,v_{i-1},u,v_{i+1},\ldots ,v_{n})}-e_{(v_{1},\ldots ,v_{i-1},w,v_{i+1},\ldots ,v_{n})}$ .

Then the residue class space ${}F/U$ is called the tensor product of the ${}V_{i}$ , ${}i\in \{1,\ldots ,n\}$ . It is denoted by

V_{1}\otimes _{K}V_{2}\otimes _{K}\cdots \otimes _{K}V_{n}.

We usually just write ${}V_{1}\otimes _{}\cdots \otimes _{}V_{n}$ . The images of ${}e_{(v_{1},\ldots ,v_{n})}$ in ${}V_{1}\otimes _{K}V_{2}\otimes _{K}\cdots \otimes _{K}V_{n}$ are denoted by

v_{1}\otimes _{}\cdots \otimes _{}v_{n}.

This is the equivalence class of ${}e_{(v_{1},\ldots ,v_{n})}$ for the equivalence relation given by the linear subspace ${}U$ . Every element in ${}V_{1}\otimes _{K}\cdots \otimes _{K}V_{n}$ has a (not unique) representation of the form

a_{1}v_{1,1}\otimes _{}\cdots \otimes _{}v_{1,n}+\cdots +a_{m}v_{m,1}\otimes _{}\cdots \otimes _{}v_{m,n}

(with ${}a_{i}\in K$ , and ${}v_{i,j}\in V_{j}$ ). In particular, the decomposable tensors ${}v_{1}\otimes _{}\cdots \otimes _{}v_{n}$ form a ${}K$ -generating system of the tensor product. The defining generators of the linear subspace become equations in the tensor product; they express the multilinearity. In particular, we have

v_{1}\otimes _{}\cdots \otimes _{}v_{i-1}\otimes rv_{i}\otimes v_{i+1}\otimes _{}\cdots \otimes _{}v_{n}=v_{1}\otimes _{}\cdots \otimes _{}v_{j-1}\otimes rv_{j}\otimes v_{j+1}\otimes _{}\cdots \otimes _{}v_{n}\,

for arbitrary ${}i,j$ , and

v_{1}\otimes _{}\cdots \otimes _{}v_{i-1}\otimes (u+w)\otimes v_{i+1}\otimes _{}\cdots \otimes _{}v_{n}=v_{1}\otimes _{}\cdots \otimes _{}v_{i-1}\otimes u\otimes v_{i+1}\otimes _{}\cdots \otimes _{}v_{n}+v_{1}\otimes _{}\cdots \otimes _{}v_{i-1}\otimes w\otimes v_{i+1}\otimes _{}\cdots \otimes _{}v_{n}\,.

Example

For ${}K=\mathbb {R}$ and ${}V_{1}=\mathbb {R} ^{2}$ and ${}V_{1}=\mathbb {R} ^{3}$ , the elements of ${}F$ (in the sense of Definition) are linear combinations like

4e_{({\begin{pmatrix}3\\2\end{pmatrix}},{\begin{pmatrix}6\\-1\\5\end{pmatrix}})}-5e_{({\begin{pmatrix}1\\7\end{pmatrix}},{\begin{pmatrix}3\\3\\4\end{pmatrix}})}+6e_{({\begin{pmatrix}2\\4\end{pmatrix}},{\begin{pmatrix}-4\\7\\8\end{pmatrix}})}.

Using the standard vectors of ${}\mathbb {R} ^{2}$ and of ${}\mathbb {R} ^{3}$ , this is

4e_{(3e_{1}+2e_{2},6e_{1}-e_{2}+5e_{3})}-5e_{e_{1}+7e_{2},3e_{1}+3e_{2}+4e_{3}}+6e_{(2e_{1}+4e_{2},-4e_{1}+7e_{2}+8e_{3})}.

Because these tuples are different, we can not simplify this expression in ${}F$ . The image of this element in ${}F/U=\mathbb {R} ^{2}\otimes _{\mathbb {R} }\mathbb {R} ^{3}$ is

4(3e_{1}+2e_{2})\otimes (6e_{1}-e_{2}+5e_{3})-5(e_{1}+7e_{2})\otimes (3e_{1}+3e_{2}+4e_{3})+6(2e_{1}+4e_{2})\otimes (-4e_{1}+7e_{2}+8e_{3}).

This expression can be simplified as a linear combination of the family ${}e_{i}\otimes e_{j}$ , ${}i=1,2$ , ${}j=1,2,3$ .

More important than the construction of the tensor product is the following universal property.

Theorem

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ denote vector spaces

over

{}K

.

The mapping
$\pi \colon V_{1}\times \cdots \times V_{n}\longrightarrow V_{1}\otimes _{K}\cdots \otimes _{K}V_{n},{\left(v_{1},\ldots ,v_{n}\right)}\longmapsto v_{1}\otimes _{}\cdots \otimes _{}v_{n},$

is ${}K$ -multilinear.
Let ${}W$ be another ${}K$ -vector space, and let
$\psi \colon V_{1}\times \cdots \times V_{n}\longrightarrow W$

denote a multilinear mapping. Then there exists a uniquely determined ${}K$ -linear mapping

${\bar {\psi }}\colon V_{1}\otimes _{K}\cdots \otimes _{K}V_{n}\longrightarrow W$

fulfilling ${}\psi ={\bar {\psi }}\circ \pi$ .

Proof

follows immediately from the definition of the tensor product.
Since the ${}v_{1}\otimes _{}\cdots \otimes _{}v_{n}$ are a ${}K$ -generating system of ${}V_{1}\otimes _{K}\cdots \otimes _{K}V_{n}$ , and because
${}{\bar {\psi }}(v_{1}\otimes _{}\cdots \otimes _{}v_{n})=\psi (v_{1},\ldots ,v_{n})\,$

must hold, there can exist at most one such s linear mapping. To show existence, we consider the ${}K$ -vector space ${}F$ from the construction of the tensor product. The ${}e_{(v_{1},\ldots ,v_{n})}$ form a basis of ${}F$ ; therefore, the assignment

${}{\tilde {\psi }}{\left(e_{(v_{1},\ldots ,v_{n})}\right)}:=\psi (v_{1},\ldots ,v_{n})\,$

defines a linear mapping

${\tilde {\psi }}\colon F\longrightarrow W.$

Because of the multilinearity of ${}\psi$ , the linear subspace ${}U$ is mapped to ${}0$ . Hence, according to the factorization theorem, this mapping induces a ${}K$ -linear mapping

${\overline {\psi }}\colon F/U\cong V_{1}\otimes _{K}\cdots \otimes _{K}V_{n}\longrightarrow W.$

\Box

Corollary

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ and ${}W$ be vector spaces over ${}K$ . Then there exists a natural isomorphism

\operatorname {Hom} _{K}{\left(V_{1}\otimes _{}\cdots \otimes _{}V_{n},W\right)}\longrightarrow \operatorname {Mult} _{}{\left(V_{1},\ldots ,V_{n};W\right)},\varphi \longmapsto \varphi \circ \pi .

Proof

From Theorem 55.4 (2), we may conclude immediately that we have a bijection.

\Box

Corollary

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ denote vector spaces over ${}K$ . Then there exists a natural isomorphism

{\left(V_{1}\otimes _{}\cdots \otimes _{}V_{n}\right)}^{*}\longrightarrow \operatorname {Mult} _{}{\left(V_{1},\ldots ,V_{n};K\right)},\varphi \longmapsto \varphi \circ \pi .

Proof

This follows immediately from Corollary 55.5 , applied to ${}W=K$ .

\Box

Remark

For ${}V=V_{1}=V_{2}$ , and ${}W=K$ , the multilinear mappings from ${}V_{1}\times V_{2}$ to ${}W$ are just the bilinear forms on ${}V$ . Corollary 55.6 says in this situation that the dual space of ${}V\otimes V$ represents all bilinear forms. For ${}V=K^{n}$ , the standard inner product (this name is only for ${}K=\mathbb {R}$ correct) corresponds to the linear form ${}\varphi \colon V\otimes V\rightarrow K$ given by

{}\varphi (e_{i}\otimes e_{j})=\delta _{ij}\,.

The tensor product is determined up to (unique) isomorphy by this universal property; this means the following.

Theorem

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ denote vector spaces over ${}K$ . Let ${}Z$ be a ${}K$ -vector space, together with a multilinear mapping

\rho \colon V_{1}\times \cdots \times V_{n}\longrightarrow Z,

and suppose that ${}(Z,\rho )$ fulfill the universal property from Theorem 55.4 (2). Then there exists a uniquely determined isomorphism

V_{1}\otimes _{}\cdots \otimes _{}V_{n}\longrightarrow Z.

Proof

Because ${}V_{1}\times \cdots \times V_{n}\rightarrow Z$ is multilinear, there exists, due to the universal property of the tensor product, a uniquely determined linear mapping

V_{1}\otimes _{}\cdots \otimes _{}V_{n}\longrightarrow Z.

Because of the universal property of ${}Z$ and the multilinearity of ${}\pi \colon V_{1}\times \cdots \times V_{n}\rightarrow V_{1}\otimes _{}\cdots \otimes _{}V_{n}$ , there is also a linear mapping

Z\longrightarrow V_{1}\otimes _{}\cdots \otimes _{}V_{n}.

Because of the universal property, they are inverse to each other.

\Box

Therefore, this universal property is more important than the explicit construction of the tensor product.

Lemma

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ denote vector spaces

over

{}K

. Then the following calculation rules hold.

For vectors ${}v_{j}\in V_{j}$ , and ${}c\in K$ , we have
$v_{1}\otimes _{}\cdots \otimes _{}v_{i-1}\otimes cv_{i}\otimes v_{i+1}\otimes _{}\cdots \otimes _{}v_{n}=c{\left(v_{1}\otimes _{}\cdots \otimes _{}v_{i-1}\otimes v_{i}\otimes v_{i+1}\otimes _{}\cdots \otimes _{}v_{n}\right)}\,.$
For vectors ${}v_{j}\in V_{j}$ , we have
${}v_{1}\otimes _{}\cdots \otimes _{}v_{i-1}\otimes 0\otimes v_{i+1}\otimes _{}\cdots \otimes _{}v_{n}=0\,$
Let ${}v_{1j},\ldots ,v_{m_{j}j}\in V_{j}$ , and ${}a_{ij}\in K$ . Then
${}{\begin{aligned}{\left(\sum _{i=1}^{m_{1}}a_{i1}v_{i1}\right)}\otimes _{}\cdots \otimes _{}{\left(\sum _{i=1}^{m_{n}}a_{in}v_{in}\right)}&=\sum _{(i_{1},\ldots ,i_{n})\in \{1,\ldots ,m_{1}\}\times \cdots \times \{1,\ldots ,m_{n}\}}a_{i_{1}1}\cdots a_{i_{n}n}v_{i_{1}1}\otimes _{}\cdots \otimes _{}v_{i_{n}n}\\\,\end{aligned}}$

holds.

Proof

(1) follows immediately from the construction. (2) follows from (1). (3) follows from the distributive law for multilinear mappings.

\Box

Example

Im ${}\mathbb {R} ^{2}\otimes _{\mathbb {R} }\mathbb {R} ^{2}$ , we have

{}{\begin{aligned}{\begin{pmatrix}5\\-7\end{pmatrix}}\otimes {\begin{pmatrix}-3\\4\end{pmatrix}}&={\left(5e_{1}-7e_{2}\right)}\otimes {\left(-3e_{1}+4e_{2}\right)}\\&=-15e_{1}\otimes e_{1}+20e_{1}\otimes e_{2}+21e_{2}\otimes e_{1}-28e_{2}\otimes e_{2}.\end{aligned}}

Lemma

Let ${}K$ be a field, and let ${}V$ denote a

{}K

-vector space. Then the following properties hold.

We have
${}V\otimes _{K}0=0\,.$
We have
${}V\otimes _{K}K=V\,,$

where ${}v\otimes s$ corresponds to the vector ${}sv$

Proof

(1) follows from Lemma 55.9 (2).

(2). The scalar multiplication

V\times K\longrightarrow V,(v,s)\longmapsto sv,

is multilinear; therefore there exists, according to Theorem 55.4 , a linear mapping

V\otimes K\longrightarrow V,v\otimes s\longmapsto sv.

This mapping is surjective, because ${}v\otimes 1$ is mapped to ${}v$ . An element in the tensor product has the form

{}\sum _{i=1}^{n}a_{i}(v_{i}\otimes s_{i})=\sum _{i=1}^{n}(a_{i}s_{i})(v_{i}\otimes 1)={\left(\sum _{i=1}^{n}a_{i}s_{i}v_{i}\right)}\otimes 1\,.

If this is mapped to ${}0$ , then we have

{}\sum _{i=1}^{n}a_{i}s_{i}v_{i}=0\,;

but then also the tensor element is ${}0$ , and the mapping is injective as well.

\Box

Theorem

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ denote vector spaces over ${}K$ . Let ${}J_{1},\ldots ,J_{n}$ be index sets, and let

v_{ij},j\in J_{i},

be vectors in

{}V_{i}

. Then the following statements hold.

If each family is a generating system of ${}V_{i}$ , then the family
$v_{1j_{1}}\otimes _{}\cdots \otimes _{}v_{nj_{n}}{\text{ with }}(j_{1},\ldots ,j_{n})\in J_{1}\times \cdots \times J_{n}$

is a generating system of ${}V_{1}\otimes _{}\cdots \otimes _{}V_{n}$ .
If each family is linearly independent in ${}V_{i}$ , then the family
$v_{1j_{1}}\otimes _{}\cdots \otimes _{}v_{nj_{n}}{\text{ with }}(j_{1},\ldots ,j_{n})\in J_{1}\times \cdots \times J_{n}$

is linearly independent in ${}V_{1}\otimes _{}\cdots \otimes _{}V_{n}$ .
If each family is a basis of ${}V_{i}$ , then the family
$v_{1j_{1}}\otimes _{}\cdots \otimes _{}v_{nj_{n}}{\text{ with }}(j_{1},\ldots ,j_{n})\in J_{1}\times \cdots \times J_{n}$

is a basis of ${}V_{1}\otimes _{}\cdots \otimes _{}V_{n}$ .

Proof

(1). Due to the construction, the decomposable tensors ${}v_{1}\otimes _{}\cdots \otimes _{}v_{n}$ form a generating system of the tensor product. Hence, it is enough to show that they are linear combinations of the given family. But this follows from Lemma 55.9 (3).

(2). We can restrict to finite families. We want to apply Lemma 14.8 . Let ${}(r_{1},\ldots ,r_{n})$ be fixed. Because of the linear independence of the families ${}v_{ij}$ in ${}V_{i}$ , there exist linear forms

\varphi _{i}\colon V_{i}\longrightarrow K

with ${}\varphi _{i}(v_{ir_{i}})\neq 0$ and with ${}\varphi _{i}(v_{ij})=0$ for ${}j\neq r_{i}$ . Therefore,

V_{1}\times \cdots \times V_{n}\longrightarrow K,(w_{1},\ldots ,w_{n})\longmapsto \varphi _{1}(w_{1})\cdots \varphi _{n}(w_{n}),

is, according to Exercise 16.39 , a multilinear mapping. Due to Corollary 55.6 , we have a corresponding linear mapping

V_{1}\otimes _{}\cdots \otimes _{}V_{n}\longrightarrow K,

which sends ${}v_{1r_{1}}\otimes _{}\cdots \otimes _{}v_{nr_{n}}$ to

{}\varphi _{1}(v_{1r_{1}})\cdots \varphi _{n}(v_{nr_{n}})\neq 0\,,

and all other elements ${}v_{1j_{1}}\cdots v_{nj_{n}}$ of the family to ${}0$ .

(3) follows from (1) and (2).

\Box

Corollary

Let ${}K$ be a field, and let ${}V_{1},\ldots ,V_{n}$ denote finite-dimensional vector spaces over ${}K$ . Then the dimension of the tensor product is

{}\dim _{K}{\left(V_{1}\otimes _{}\cdots \otimes _{}V_{n}\right)}=\dim _{K}{\left(V_{1}\right)}\cdots \dim _{K}{\left(V_{n}\right)}\,.

Proof

This follows immediately from Theorem 55.12 (3).

\Box

Remark

We relate the motivating Example 55.1 with the general construction of the tensor product. The mapping (with the direct meaning of ${}f\otimes g$ from the example)

\operatorname {Map} \,{\left(I,K\right)}\times \operatorname {Map} \,{\left(J,K\right)}\longrightarrow \operatorname {Map} \,{\left(I\times J,K\right)},(f,g)\longmapsto f\otimes g,

is, due to Exercise 55.5 , multilinear. Because of Theorem 55.4 (2), there exists a uniquely determined linear mapping

\operatorname {Map} \,{\left(I,K\right)}\otimes _{K}\operatorname {Map} \,{\left(J,K\right)}\longrightarrow \operatorname {Map} \,{\left(I\times J,K\right)},

where the tensor products correspond to each other. The surjectivity follows from the fact that the basis elements ${}e_{(i,j)}$ are in the image. The injectivity follows from Corollary 55.13 .

<< \| Linear algebra (Osnabrück 2024-2025)/Part II \| >> PDF-version of this lecture Exercise sheet for this lecture (PDF)