Tensors

Linear algebra review

Matrices’ columns are transformed versions of your basic vectors. Linear transformation = function that takes vector as input and outputs vector = matrix-vector multiplication = linear combination of the transformed basic vectors

If the transformed basis vectors are linearly dependent, they squish all of 2D space into a straight line.

Linear transformations must keep the origin fixed and grid lines remain straight and evenly spaced.

Matrices are transformations of space.

Linear transformation with the original basis vectors:

Broadcasting

Respect broadcasting semantics!

Two shapes are broadcast-compatible if, starting from the trailing dimensions (the rightmost), they either:

1. are equal, or
2. one of them is 1.

If one of them is 1, that dimension can be “stretched” (broadcast) to match the other.

If neither rule holds, broadcasting fails.

Intuition: You can stretch a dimension of size 1 to match the other shape. But you can’t stretch a dimension of size 2 to match size 3.

Distance measures

Dot product: Measures the similarity of two vectors by multiplying their corresponding elements and summing the results. Higher values indicate more similarity. How much do the vectors point in the same direction?

Cosine similarity: Measures the cosine of the angle between two vectors. Ranges from -1 (opposite directions) to 1 (same direction). Useful for measuring similarity between vectors regardless of their magnitude. Do two vectors point in the same direction?

When vectors are normalized to unit length, cosine similarity is equivalent to the dot product.

L2 distance (Euclidean distance): Measures the straight-line distance between two vectors in space. Calculated as the square root of the sum of squared differences between corresponding elements. What is the length of the vector connecting the tips of two vectors?