Tensors

Linear algebra review

Matrices’ columns are transformed versions of your basic vectors. Linear transformation = function that takes vector as input and outputs vector = matrix-vector multiplication = linear combination of the transformed basic vectors

If the transformed basis vectors are linearly dependent, they squish all of 2D space into a straight line.

Linear transformations must keep the origin fixed and grid lines remain straight and evenly spaced.

Matrices are transformations of space.

Linear transformation with the original basis vectors:

Broadcasting

Respect broadcasting semantics!

Two shapes are broadcast-compatible if, starting from the trailing dimensions (the rightmost), they either:

1. are equal, or
2. one of them is 1.

If one of them is 1, that dimension can be “stretched” (broadcast) to match the other.

If neither rule holds, broadcasting fails.

Intuition: You can stretch a dimension of size 1 to match the other shape. But you can’t stretch a dimension of size 2 to match size 3.