Serlo: EN: Linear independence

{{#invoke:Mathe für Nicht-Freaks/Seite|oben}}

Motivation Vorlage:Anker

Basic motivation

Maybe, you learned about vectors in school, where they were drawn as arrows in the plane or in space. Both a plane and space are vector spaces. But how do they differ?

A spontaneous answer could be: "The plane is two-dimensional and the space is three-dimensional". But this brings us immediately to further questions:

• What is the dimension of a vector space?
• How can we define it?

In the definition of the vector space the term "dimension" does not occur...

Intuition of a dimension

A sphere is a three-dimensional object

The term "dimension" describes in how many independent directions geometric objects can be extended in a space. The objects can also move in just as many independent directions in space ("degrees of freedom of motion").

The plane has two dimensions - the width and the length. It is flat, no object of the plane can reach out of it "into height". A sphere as a three-dimensional object cannot be part of the plane. In contrast, the space with length, width and height has three dimensions. A sphere can thus be part of space.

We summarize: The dimension intuitively corresponds to the number of independent directions into which a geometric object can expand or move. So, for the definition of dimension, we need to answer the following questions:

• What is a direction in a vector space?
• When are two directions independent?
• How can the number of independent directions be determined?

Derivation of the Definition

What is a direction within a vector space?

Let's take the vector space of the plane as an example. We can represent a direction with an arrow:

Pfeil, der eine Richtung in der Ebene markiert

Now an arrow is nothing but a vector. So with the help of vectors, directions can be represented. Here we must not use the zero vector. As an arrow of length zero it has no direction. We can generalize this to arbitrary vector spaces:

The direction in which the vector points is $\alpha \cdot v:\alpha \in ℝ\}$, that is, the span $\mathrm{span}(\{v\})$ of the vector $v$. To this span belong all extensions $\alpha v$ of the direction vector $v$ and thus describes the straight line, which is spanned by $v$:

A straight line described by the vector v

From the line to the plane

To get from the straight line to the plane, we need not only one vector but several, more precisely at least two vectors ($v,w$). This is intuitively obvious, because a plane can be spanned unambiguously only with two vectors. Therefore we need a further, linearly independent vector. What does "independent" mean in this case? First, we notice that the new vector must not be the zero vector. This vector does not give any direction. Furthermore, the new vector must also not be a multiple of the original vector, i.e. $w\ne \alpha v$ . This also holds for reflections of straight line vectors, represented by multiplicatioin with a negative factor.

• 
  The vector $w$ is a stretched version of the vector $v$ with a positive stretching factor. This vector $w$ is not pointing in a direction independent of $v$ .
• 
  The vector $w$ is a stretched version of the vector $v$ with a positive stretching factor (which includes a reflection). This vector $w$ is not pointing in a direction independent of $v$.
• 
  The direction of $w$ is independent of $v$. Both vectors together span a plane.

We conclude: The new vector $w$ is independent of the direction vector $v$ exactly when the latter is not on the straight line. So we need $w\ne \alpha v$ for all real numbers $\alpha$. Hence, the new vector must not be in the span of the other one. The two spans have only the zero point as intersection.

From the plane to space

We have just seen that we can characterize a plane by two independent vectors. Now we want to go from the plane to space. Here, we also have to add an independent direction. But what is a direction independent of the plane?

The new vector must not be the zero vector, because this vector does not indicate a direction. The new vector must also not lie in the plane, because in that case, no new direction would be described. Only if the new vector does not lie in the plane, it will point in a new, independent direction:

• 
  The vector $c$ lies in the plane spanned by $a$ and $b$. Hence, $c$ does not point into a direction spanned by $a$ and $b$.
• 
  The vector $u$ does not lie in the plane spanned by $v$ and $w$ . All three vectors span the entire space, which means that $u$ points into a direction independent of $v$ and $w$ .

How can we formulate this insight mathematically? Let $v$ and $w$ be the two direction vectors spanning the plane. This plane is then equal to the set $\{\alpha v+\beta w:\alpha ,\beta \in ℝ\}$. Hence, the plane is the set of all sums $\alpha v+\beta w$ for real numbers $\alpha ,\beta \in ℝ$. In order for the new vector $u$ not to be in the plane, it must be $u\ne \alpha v+\beta w$ for all $\alpha ,\beta \in ℝ$. Thus, $u$ in independent of <ma

A first criterion for linear independence

Let's summarize: To describe a straight line we needed a vector $v$ not being the zero vector. In the transition from the straight line to the plane, we had to add a vector $v$ independent of $w$. Independence of $w$ from $v$ means that $w$ does not lie in the line described by $v$ . So we need to have $w\ne \alpha v$ for all $\alpha \in ℝ$.

In the second step, we added a new direction $u$ to the plane, which is independent of the two vectors $v$ and $w$. Here independence manifests itself in the fact that $u$ is not in the plane spanned by $v$ and $w$ . Hence, we need $u\ne \alpha v+\beta w$ for all real numbers $\alpha$ and $\beta$. We can generalise this to an arbitrary number of vectors (but it is not so easy to visualize anymore):

In the above description, the sum ${\lambda }_1{v}_1+{\lambda }_2{v}_2+\dots +{\lambda }_n{v}_n$ appears. Such a sum is called linear combination of the vectors $v_1$ to $v_n$ . We may also say that $w$ is linearly independent if $w\notin \mathrm{span}\{v_1,...,v_n\}$. The description can be changed to:

Here we have clarified when a vector is independent of other vectors. Is this sufficient to describe the independence of vectors?! Take the following three vectors $a$, $b$ and $c$ as an example:

Three vectors lying in one plane

Since no vector is a multiple of another vector, the three vectors, seen in pairs, point in independent directions. For example $a$ is independent of $b$ and $c$is independent of $a$. So the three vectors are not independent of each other because they all lie in one plane. We have $c=a+b$ and so $c$ is independent of $a$ and $b$. Accordingly, we have to impose linear independence between $a$, $b$ and $c$:

• $a$ is independent of $b$ and $c$: We have $a\ne \beta b+\gamma c$ for all $\beta ,\gamma \in ℝ$.
• $b$ is independent of $a$ and $c$: We have $b\ne \alpha a+\gamma c$ for all $\alpha ,\gamma \in ℝ$.
• $c$ is independent of $a$ and $b$: We have $c\ne \alpha a+\beta b$ for all $\alpha ,\beta \in ℝ$.

It should be emphasised at this point that it is necessary to require all three conditions. If we were to waive the last two conditions, the first requirement would guarantee that the vector $a$ is linearly independent of the vectors $b$ and $c$, but it is not clear from this requirement that $b$ and $c$ are linearly independent of each other. This does not have to be fulfilled, which would mean that the three vectors would again not be linearly independent of each other.

Therefore, none of the three vectors must be able to be represented as a linear combination of the other two vectors. Otherwise at least one of the vectors is dependent on the other vectors. We can generalise this to any number of vectors:

Mathe für Nicht-Freaks: Vorlage:Definition

From the first criterion to the formal definition

With our first criterion, which we found above, we have already found a suitable definition for linear independence of vectors. In the following, we will try to find a more concise equivalent criterion, with which we can examine the linear independence of vectors more easily.

Vectors are independent if no vector can be represented as a linear combination of the other vectors. From this we will derive another criterion for linear independence, which is less computationally demanding. Let us take vectors $v_1$, $v_2$ to $v_n$ from a vector space $V$ that are not independent. So there is one vector that can be represented by the others. Let $v_1$ be this vector. There are thus stretching factors (scalars) ${\lambda }_2$ to ${\lambda }_n$, such that Vorlage:Einrücken We can transform this equation by computing $-v_1$ on both sides ($0_V$ is the zero vector of the vector space $V$): Vorlage:Einrücken This is a so-called nontrivial linear combination of the zero vector. A nontrivial linear combination of the zero vector is a linear combination with the result $0_V$ where at least one coefficient is not equal to $0$. For ${\lambda }_1={\lambda }_2=\dots ={\lambda }_n=0$ we would trivially ${\lambda }_1{v}_1+{\lambda }_2{v}_2+{\lambda }_3{v}_3+\dots +{\lambda }_n{v}_n=0_V$. This is the so-called trivial linear combination of the zero vector, where all coefficients are equal to $0$. You can always form this trivial linear combination, no matter which vectors $v_1$ to $v_n$ you choose. So it does not carry information. If $v_1$ to $v_n$ are dependent, there is at least one non-trivial linear combination of the zero vector (as we saw above) in addition to the trivial linear combination. So:

In other words: Vorlage:Important Now we can apply the principle of contraposition: $A⟹B$ holds if and only if $\mathrm{\neg }B⟹\mathrm{\neg }A$. So:

With this we have found a criterion for linear independence. If the zero vector can only be represented trivially by a linear combination of $v_1$ to $v_n$, then these vectors are linearly independent. However, this criterion can also be used as a definition of linear independence. To do this, we need to show the converse direction of the above implication. If there is a non-trivial linear combination of the zero vector, then the vectors under consideration are linearly dependent.

So let $v_1$ to $v_n$ be vectors for which there exists a non-trivial linear combination of the zero vector. This means, there are coefficients (scalars) ${\lambda }_1$ to ${\lambda }_n$, such that ${\lambda }_1{v}_1+\dots +{\lambda }_n{v}_n=0_V$ where at least one of the coefficients ${\lambda }_1$ to ${\lambda }_n$ is not $0$. Let ${\lambda }_i$ be this coefficient. Then

Since ${\lambda }_i\ne 0$ we can multiply both sides by $-{\lambda }_i^{-1}=-\frac{1}{{\lambda }_i}$ . Then,

On both sides we can now add $v_i$:

Thus $v_i$ can be represented as a linear combination of the other vectors and hence the vectors $v_1$ to $v_n$ are linearly dependent. This proves taht the following definition of linear independence is equivalent to the first one:

Definition of a family

We have talked above about a several vectors $v_1,\dots ,v_n$ being linearly independent. But what is this "collection" of vectors $v_1,\dots ,v_n$ from a mathematical point of view? We already know the notion of a set. So it is obvious to understand $M=\{v_1,\dots ,v_n\}$ also as a set. Does this view intuitively fit linear independence? Actually, it turns out problematic, if we have two equal vectors $v,v$ with $v\ne 0$. Both point in the same direction and span no two independent directions. Thus they are intuitively linearly dependent. And indeed, one can be written as a linear combination of the other as $v=1\cdot v$. Thus the vectors $v,v$ are also strictly mathematically linearly dependent. However, a set may only contain different elements. That is, the set containing $v$ and $v$ is $M=\{v,v\}=\{v\}$. So the set $M$ contains only one element and does not capture duplications of vectors.

So we need a new mathematical term that also captures duplications. This is the concept of family:

Mathe für Nicht-Freaks: Vorlage:Definition

Formally, a family can be seen as a mapping of the index set $I$ into the set $A$. In contrast to sets, elements may occur more than once in families, namely if they belong to different indices.

If the set $I$ is countable, the elements of the family can be numbered: $(a_1,a_2,\dots )$. However, the index set $I$ may also be overcountable, e.g. $I=ℝ$. In this case $(a_i{)}_{i\in ℝ}$ cannot be written as a sequence $(a_1,a_2,\dots )$. The term family thus contains all sequences, and includes even larger "collections" of mathematical objects.

So when we say the vectors $v$ and $v$ are linearly dependent we can express it by saying that the family $(v_i{)}_{i\in \{1,2\}}$ with $v_1=v_2=v$ is linearly dependent.

Often one writes (with slight abuse of notation) $(a_i)\subseteq A$ if the $a_i$ are elements of $A$ and it is clear from the context what the index set $I$ looks like. Similarly, $a\in (a_i)$ means that there is an $i\in I$ with $a_i=a$.

With this we can rewrite the second definition of linear independence:

Mathe für Nicht-Freaks: Vorlage:Definition

General definition of linear independence

Motivation

We have learned above two definitions for the fact that finitely many vectors $v_1,\dots ,v_n\in V$ are linearly independent:

1. A somewhat unwieldy: vectors are independent if no vector $v_k$ can be written as a linear combination of the others. So $v_k={\lambda }_1{v}_1+\dots +{\lambda }_{k-1}{v}_{k-1}+{\lambda }_{k+1}{v}_{k+1}+\dots +{\lambda }_n{v}_n$ must not occur.
2. A somewhat more compact one: The zero vector $0_V$ can only be represented as a trivial linear combination. So $0_V={\lambda }_1{v}_1+\dots +{\lambda }_n{v}_n$ implies ${\lambda }_1=\dots ={\lambda }_n=0$.

So far we have only considered finitely many vectors. What happens with infinitely many vectors? Can there even be an infinite number of linearly independent vectors? We would need a vector space that has infinitely many linearly independent directions. We know intuitively that the vector space ${ℝ}^2$ has at most two and the ${ℝ}^3$ at most three independent directions. So we need a much "bigger" vector space to get infinitely many independent directions. So we consider a vector space $V$ where every vector has infinitely many coordinates: $v=(x_1,x_2,\dots )$ with $x_1,x_2,\dots \in ℝ$. Accordingly, $v$ corresponds to a real sequence $(x_i{)}_{i\in \mathbb{N}}$ and $V$ is the sequences vector space, or sequence space.

In ${ℝ}^d$ we have the linearly independent unit vectors $(1,0,\dots ,0),(0,1,\dots ,0),\dots ,(0,\dots ,0,1)$. We can continue this construction and obtain for $i\in \mathbb{N}$ the vectors $e_i=(0,\dots ,0,1,0,\dots )$ with the $1$ at the $i$-th place and otherwise $0$.

The infinitely many vectors $e_1,e_2,\dots$ form a family $(e_i{)}_{i\in \mathbb{N}}$. This family intuitively represents "infinitely many different directions" in $V$ and is thus intuitively linearly independent. So it makes sense to define linear independence for infinitely many vectors in such a way that $(e_i{)}_{i\in \mathbb{N}}$ is a linearly independent family. The "somewhat unwieldy definition 1." above would be suitable for this in principle: We could simply copy it and say "a family of vectors $(v_i{)}_{i\in \mathbb{N}}$ is linearly independent if no $v_i$ can be written as a linear combination of the others". In fact, in $(e_i{)}_{i\in \mathbb{N}}$ none of the $e_i$ can be written as a linear combination of the other vectors. Therefore, the definition already makes sense at this point. However, there are infinitely many $e_i$ and thus infinitely many conditions!

We prefer to consider the "somewhat more compact definition 2.": "Vectors $(v_i{)}_{i\in \mathbb{N}}$ are linearly independent if $0_V$ can only be represented by the trivial linear combination." What does this formulation mean explicitly in this example? We are given a linear combination of $0_V\in V$. Linear combinations are finite, that is, we have finitely many vectors $e_{i_1},\dots ,e_{i_n}$ and ${\lambda }_1,\dots ,{\lambda }_n\in ℝ$ such that

We now have to show that all ${\lambda }_i=0$, since then the linear combination of $0_V\in V$ above is trivial. This works in exactly the same way as in ${ℝ}^d$, except that here we have to compare infinitely many entries.

What do we have to do now to get a general definition for general families and general vector spaces? The "somewhat more compact definition 2." carries over almost literally: "A family $(v_i{)}_{i\in I}$ of vectors is linearly independent if $0_V$ can only be represented by the trivial linear combination." For the written out implication, we can make use of our language of families: We replace the double indices by the word "sub-family".

Implications of the definition

Re-formulating the definition for finite sub-families Vorlage:Anker

We have a definition of linear independence for arbitrary subfamilies of a vector space $V$. Does this agree with our old definition for finite subfamilies? Intuitively, they should agree for finite subfamilies, since we derived the general definition from our old definition. The following theorem actually proves this:

Reducing the definition to finite sub-families

We have defined linear independence for any family $(v_i{)}_{i\in I}$ of vectors, so also for infinitely many vectors. But in the definition we only need to show a statement for finite subfamilies $(v_j{)}_{j\in J}$: For all ${\lambda }_j\in K$ with $j\in J$ we need the following: Vorlage:Einrücken In the previous theorem we have seen that this statement is exactly linear independence of $(v_j{)}_{j\in J}$.

Overview

The following properties can be derived from the definition of linear independence with a few proof steps. Let $K$ be a field and $V$ a $K$-vector space:

1. Every sub-family of a family of linearly independent vectors is linearly independent. Conversely, every super-family of a family of linearly dependent vectors is again linearly dependent.
2. Let $v\in V$ be a single vector. Then $v$ is linearly independent if and only if $v\ne 0_V$. So "almost always". Conversely, every family (no matter how large) is linearly dependent as soon as it contains the zero vector.
3. Let $v,w\in V$. The vectors $v$ and $w$ are linearly dependent if and only if there is a $\lambda \in K$ with the property $w=\lambda \cdot v$ or $v=\lambda \cdot w$.
4. If a family of vectors is linearly dependent, one of them can be represented as a linear combination of the others.

Sub-families of linear independent vectors are linearly independent

A linearly independent family remains linearly independent if you take away vectors. Linear dependence, on the other hand, is preserved if you add more vectors. Intuitively, the addition of vectors tends to "destroy" linear independence and cannot be restored by adding more vectors.

Families including the zero vector are linearly independent

When is a family with exactly one vector linearly independent? This question is easy to answer: whenever this vector is not the zero vector. Conversely, every family with the zero vector is linearly dependent. Including the one that contains only the zero vector itself.

Two vectors are linearly dependent if one is a stretched version of the other

When is a family with two vectors linearly independent? We can answer the question by saying when the opposite is the case. So when are two vectors linearly dependent? Linear dependence of two vectors holds if and only if both "lie on a straight line", i.e. one vector is a stretched version of the other.

With linear dependence, one vector is a linear combination of the others

For finitely many vectors, we started with the definition that vectors are linearly dependent if one of the vectors can be written as a linear combination of the others (first definition). We have already seen that this definition is equivalent to the null vector being able to be written as a linear combination of the vectors (second definition). For the general definition with possibly infinitely many vectors, we have used the version with the zero vector (the second) as our definition. And one can indeed show that even in the general case the first definition is equivalent to it:

Linear independence and unique linear combinations Vorlage:Anker

In this section, we will take a closer look at the connection between linear independence and linear combinations. To do this, we recall what it means that the vectors $v_1,\dots ,v_n$ are linearly dependent or independent. Suppose the vectors $v_1,\dots ,v_n$ are linearly dependent. From our definition of linear independence, we know that there must then be a non-trivial zero representation, since at least one scalar ${\lambda }_i\ne 0$ for some $1\le i\le n$. We illustrate this with the following example

Mathe für Nicht-Freaks: Vorlage:Beispiel

Regardless of whether the considered family of vectors is linearly independent or not, there is always the trivial zero representation, in which all scalars ${\lambda }_1,...,{\lambda }_n$ have the value $0$:

In case of linear dependence of the vectors, the representation of the zero is no longer unambiguous. We can summarise our results so far in a theorem and generalise them:

Exercises

Exercise 6

{{#invoke:Mathe für Nicht-Freaks/Seite|unten}}