Basis and Dimension

Once we understand vector spaces, the next critical question is: how many vectors do we really need to describe everything inside that space?

The ideas of basis and dimension answer this question precisely. They form the backbone of linear algebra, data representation, compression, and machine learning models.

---

Why Basis and Dimension Are Important

In real life, efficiency matters.

Basis and dimension help us:

• Represent data using the smallest number of vectors
• Understand degrees of freedom
• Reduce complexity in models
• Explain how much information a system holds

Modern AI depends heavily on these ideas.

---

What Is a Basis?

A basis of a vector space is a set of vectors that:

• Are linearly independent
• Span the entire vector space

Using a basis, every vector in the space can be written uniquely as a combination of basis vectors.

---

Intuition Behind a Basis

Think of a basis as the building blocks of a space.

Just like:

• Two directions describe a plane
• Three directions describe 3D space

A basis gives the minimum directions needed to describe everything.

---

Standard Basis of ℝ²

The standard basis of ℝ² consists of:

e₁ = (1, 0),
e₂ = (0, 1)

Any vector (x, y) can be written as:

(x, y) = x·e₁ + y·e₂

This shows how basis vectors generate the space.

---

Standard Basis of ℝ³

The standard basis of ℝ³ is:

e₁ = (1, 0, 0),
e₂ = (0, 1, 0),
e₃ = (0, 0, 1)

Three independent directions define 3D space.

---

What Does “Span” Mean?

A set of vectors spans a vector space if every vector in the space can be written as a linear combination of them.

If vectors do not span the space, some directions are missing.

---

What Does “Linearly Independent” Mean?

Vectors are linearly independent if none of them can be written as a combination of the others.

If one vector is redundant, the set is not independent.

A basis must contain no redundancy.

---

Example: Linearly Dependent Set

Consider the vectors:

(1, 0), (2, 0)

These are linearly dependent because (2, 0) = 2·(1, 0).

They cannot form a basis for ℝ².

---

Example: Linearly Independent Set

The vectors:

(1, 0), (0, 1)

are linearly independent and span ℝ².

So they form a basis of ℝ².

---

Definition of Dimension

The dimension of a vector space is the number of vectors in any basis of that space.

It measures:

• Degrees of freedom
• Number of independent directions

Dimension tells us how “big” a space is.

---

Dimension of Common Vector Spaces

Examples:

• ℝ¹ → dimension 1
• ℝ² → dimension 2
• ℝ³ → dimension 3
• ℝⁿ → dimension n

This idea extends to very high dimensions.

---

Important Property of Dimension

All bases of a vector space have the same number of vectors.

This is a fundamental theorem of linear algebra.

So dimension is well-defined.

---

Basis Is Not Unique

A vector space can have many different bases.

However:

• All bases have the same size
• All describe the same space

Different bases give different perspectives.

---

Basis and Coordinate Representation

Once a basis is chosen, every vector can be represented by coordinates relative to that basis.

Changing the basis changes the coordinates, not the vector itself.

This idea is crucial in transformations.

---

Basis and Dimension in Geometry

Geometrically:

• A line → dimension 1
• A plane → dimension 2
• Space → dimension 3

Basis vectors describe directions of extension.

---

Basis and Dimension in Physics

Physics uses basis vectors to describe:

• Coordinate systems
• Motion directions
• State spaces

Changing basis simplifies equations.

---

Basis and Dimension in Data Science

In data science:

• Each feature represents a dimension
• Datasets live in high-dimensional spaces

Dimensionality reduction aims to find better bases with fewer dimensions.

---

Basis and Dimension in Machine Learning

Machine learning models work in:

• Feature spaces
• Embedding spaces
• Latent spaces

Reducing dimension improves efficiency and performance.

---

Basis and Dimension in Competitive Exams

Exams often test:

• Definition of basis
• Linear independence
• Finding dimension

Clear logic is more important than formulas.

---

Common Mistakes to Avoid

Students often make these mistakes:

• Including dependent vectors in a basis
• Confusing span with independence
• Assuming basis is unique

A valid basis must satisfy all conditions.

---

Practice Questions

---

Quick Quiz

---

Quick Recap

• A basis is a minimal spanning set
• Basis vectors must be independent
• Dimension equals number of basis vectors
• All bases have the same size
• Foundation for eigenvalues and SVD

With basis and dimension mastered, you are now ready to learn eigenvalues and eigenvectors, where linear algebra meets deep structure.