Vectors – Vector Transposition, Norms, and Unit Vectors: A Comprehensive Guide
In the world of data science, vectors play a vital role. They are fundamental to machine learning, data analysis, and artificial intelligence.
This post aims to break down the concepts of vectors, vector transposition, norms, and unit vectors, explaining their importance in the data science landscape.
Vectors
Vectors, also referred to as first-order tensors, are mathematical objects used to represent quantities that have both magnitude and direction. They are an ordered list of numbers that can describe anything from motion to the weights in a neural network.
$ F = \begin{bmatrix}
v1 \\
v2 \\
v3 \\
\end{bmatrix} $
or
$ F = \begin{bmatrix}
v1 & v2 & v3 \
\end{bmatrix} $
Properties of Vectors
- Commutativity: A + B = B + A
-
Associativity: (A + B) + C = A + (B + C)
-
Distributive Property: a(A + B) = aA + aB, where ‘a’ is a scalar.
1. Basic Operations with Vectors
-
Addition: When you add vectors, you place them head to tail and draw the resultant vector from the tail of the first vector to the head of the last.
-
Subtraction: This is equivalent to adding a negative vector.
-
Scalar Multiplication: Multiplying a vector by a scalar changes its magnitude but not its direction.
-
Dot Product (Scalar Product): The result is a scalar. For two vectors A and B, A·B = |A||B|cos(θ), where θ is the angle between the vectors.
-
Cross Product (Vector Product): The result is a vector. For vectors A and B in 3D, the magnitude is |A||B|sin(θ), and the direction is perpendicular to both A and B, following the right-hand rule.
Addition of Vectors
python
import numpy as np
import torch
import tensorflow as tf
# NumPy
v1 = np.array([1,2,3])
v2 = np.array([4,5,6])
result = v1 + v2
print("NumPy result : {}".format(result))
# PyTorch
v1 = torch.tensor([1,2,3])
v2 = torch.tensor([4,5,6])
result = v1 + v2
print("PyTorch result : {}".format(result))
# TensorFlow
v1 = tf.constant([1,2,3])
v2 = tf.constant([4,5,6])
result = v1 + v2
print("TensorFlow result : {}".format(result))
python
NumPy result : [5 7 9]
PyTorch result : tensor([5, 7, 9])
TensorFlow result : [5 7 9]
Subtraction of Vectors
python
# NumPy
v1 = np.array([1,2,3])
v2 = np.array([4,5,6])
result = v1 - v2
print("NumPy result : {}".format(result))
# PyTorch
v1 = torch.tensor([1,2,3])
v2 = torch.tensor([4,5,6])
result = v1 - v2
print("PyTorch result : {}".format(result))
# TensorFlow
v1 = tf.constant([1,2,3])
v2 = tf.constant([4,5,6])
result = v1 - v2
print("TensorFlow result : {}".format(result))
python
NumPy result : [-3 -3 -3]
PyTorch result : tensor([-3, -3, -3])
TensorFlow result : [-3 -3 -3]
Scalar Multiplication
python
# NumPy
v = np.array([1,2,3])
result = 2 * v
print("NumPy result : {}".format(result))
# PyTorch
v = torch.tensor([1,2,3])
result = 2 * v
print("PyTorch result : {}".format(result))
# TensorFlow
v = tf.constant([1,2,3])
result = 2 * v
print("TensorFlow result : {}".format(result))
python
NumPy result : [2 4 6]
PyTorch result : tensor([2, 4, 6])
TensorFlow result : [2 4 6]
Dot Product (Scalar Product)
python
# NumPy
v1 = np.array([1,2,3])
v2 = np.array([4,5,6])
result = np.dot(v1, v2)
print("NumPy result : {}".format(result))
# PyTorch
v1 = torch.tensor([1,2,3])
v2 = torch.tensor([4,5,6])
result = torch.dot(v1, v2)
print("PyTorch result : {}".format(result))
# TensorFlow
v1 = tf.constant([1,2,3])
v2 = tf.constant([4,5,6])
result = tf.tensordot(v1, v2, axes=1)
print("TensorFlow result : {}".format(result))
python
NumPy result : 32
PyTorch result : 32
TensorFlow result : 32
Cross Product (Vector Product)
python
# NumPy
v1 = np.array([1,2,3])
v2 = np.array([4,5,6])
result = np.cross(v1, v2)
print("NumPy result : {}".format(result))
# PyTorch
v1 = torch.tensor([1,2,3])
v2 = torch.tensor([4,5,6])
result = torch.cross(v1, v2)
print("PyTorch result : {}".format(result))
python
NumPy result : [-3 6 -3]
PyTorch result : tensor([-3, 6, -3])
2. Vector Transposition
Transposing a vector means flipping its orientation. If you have a row vector, transposing it will turn it into a column vector, and vice versa.
The transposition of vector V is denoted by $V^{T}$
Transposition is commonly used in matrix multiplication and finding correlations between variables.
$ V = \begin{bmatrix}
v1 \\
v2 \\
v3 \\
\end{bmatrix} $
$ V^{T}= \begin{bmatrix}
v1 &
v2 &
v3 \
\end{bmatrix}$
Transposition is commonly used in matrix multiplication and finding correlations between variables.
python
# NumPy
vector = np.array([[1, 2, 3]])
transposed_vector = vector.T
print("Row Vector : {}".format(vector))
print("NumPy result : {}".format(transposed_vector))
# PyTorch
vector = torch.Tensor([[1, 2, 3]])
transposed_vector = vector.T
print("PyTorch result : {}".format(transposed_vector))
# TensorFlow
vector = tf.constant([[1, 2, 3]])
transposed_vector = tf.transpose(vector)
print("TensorFlow result : {}".format(transposed_vector))
python
Row Vector : [[1 2 3]]
NumPy result : [[1]
[2]
[3]]
PyTorch result : tensor([[1.],
[2.],
[3.]])
TensorFlow result : [[1]
[2]
[3]]
3. Norms
The norm of a vector provides a measure of its length or magnitude. In data science, norms are useful for understanding the scale of vectors, like the weights in a neural network or the error in a regression model.
L1 and L2 Norms
The two most common norms are the L1 and L2 norms:
L1 Norm: Sum of absolute values of the components.
$∣∣V∣∣1 = ∣v1| + ∣v2∣ + ∣v3∣$
L2 Norm: Square root of the sum of squared components (Euclidean norm).
$∣∣V∣∣2 = \sqrt[]{v_1^2 + v_2^2 +v_3^2}$
python
# Numpy
vec = np.array([1, -2, 3])
# L1 norm
l1_norm_numpy = np.linalg.norm(vec, ord=1)
print(f"L1 norm using numpy: {l1_norm_numpy}")
# L2 norm
l2_norm_numpy = np.linalg.norm(vec, ord=2)
print(f"L2 norm using numpy: {l2_norm_numpy}")
python
L1 norm using numpy: 6.0
L2 norm using numpy: 3.7416573867739413
python
# PyTorch
vec_torch = torch.tensor([1, -2, 3], dtype=torch.float32)
# L1 norm
l1_norm_pytorch = torch.norm(vec_torch, p=1)
print(f"L1 norm using PyTorch: {l1_norm_pytorch.item()}")
# L2 norm
l2_norm_pytorch = torch.norm(vec_torch, p=2)
print(f"L2 norm using PyTorch: {l2_norm_pytorch.item()}")
python
L1 norm using PyTorch: 6.0
L2 norm using PyTorch: 3.7416574954986572
python
# TensorFlow
vec_tf = tf.constant([1, -2, 3], dtype=tf.float32)
# L1 norm
l1_norm_tensorflow = tf.norm(vec_tf, ord=1)
print(f"L1 norm using TensorFlow: {l1_norm_tensorflow.numpy()}")
# L2 norm
l2_norm_tensorflow = tf.norm(vec_tf, ord=2)
print(f"L2 norm using TensorFlow: {l2_norm_tensorflow.numpy()}")
python
L1 norm using TensorFlow: 6.0
L2 norm using TensorFlow: 3.7416574954986572
4. Unit Vectors
A unit vector is a vector with a length or norm of 1. Unit vectors are important for providing direction without a specific magnitude. They are used in various data science applications, such as feature scaling and direction cosines.
To convert a vector into a unit vector, divide it by its magnitude:
$\hat{V} = \frac{V}{||V||}$
python
# Numpy
# Create a vector
vector = np.array([1, 2, 3])
# Normalize the vector to get the unit vector
unit_vector = vector / np.linalg.norm(vector)
print("NumPy Unit Vector:", unit_vector)
python
NumPy Unit Vector: [0.26726124 0.53452248 0.80178373]
python
# PyTorch
# Create a vector
vector = torch.tensor([1.0, 2.0, 3.0])
# Normalize the vector to get the unit vector
unit_vector = vector / torch.norm(vector)
print("PyTorch Unit Vector:", unit_vector)
python
PyTorch Unit Vector: tensor([0.2673, 0.5345, 0.8018])
python
# TensorFlow
# Create a vector
vector = tf.constant([1.0, 2.0, 3.0])
# Normalize the vector to get the unit vector
unit_vector = vector / tf.norm(vector)
# For TensorFlow 2.x:
print("TensorFlow Unit Vector:", unit_vector.numpy())
python
TensorFlow Unit Vector: [0.26726124 0.5345225 0.8017837 ]
Conclusion
Vectors, or first-order tensors, are foundational to understanding higher-dimensional tensors and are omnipresent in various scientific computations. Their operations, such as transposition, finding norms, and deriving unit vectors, are fundamental to advanced mathematics and physics.
Understanding these concepts is pivotal for anyone diving into linear algebra, data science, quantum mechanics, and numerous other fields.
Free Course
Master Core Python — Your First Step into AI/ML
Build a strong Python foundation with hands-on exercises designed for aspiring Data Scientists and AI/ML Engineers.
Start Free Course →Trusted by 50,000+ learners
Related Course
Master Linear Algebra — Hands-On
Join 5,000+ students at edu.machinelearningplus.com
Explore Course
Related Articles
