promptdojo_

Tensors and shapes before anything else — step 1 of 7

Tensors and shapes: PyTorch's one data structure

PyTorch has exactly one core data structure: the tensor. It is chapter 35's ndarray with two superpowers bolted on — it can live on a GPU, and it can remember how it was computed (autograd, lesson 03). Everything else in deep learning is functions over tensors.

Browser note: torch doesn't run in this editor, so the runnable demos model tensors as nested lists — which is exactly what they are, structurally. The fenced snippets are the real torch calls for your machine.

Shape is the language

Run the editor. A tensor's shape is the tuple of its dimension sizes, and reading shapes is reading deep-learning code:

import torch
x = torch.tensor([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
x.shape        # torch.Size([3, 2])
x.dtype        # torch.float32 — the default for model math
x.device       # cpu (or cuda:0 when it lives on a GPU)

The recurring shapes you'll meet, worth memorizing as words:

  • (batch, features) — classic tabular input, chapter 39's X.
  • (batch, seq_len, dim) — a batch of token sequences, each token an embedding. The transformer's native shape (chapter 44).
  • (batch, channels, height, width) — images in torch's convention.

When you read x.mean(dim=0), the question is always "which axis am I collapsing?" — dim=0 averages across the batch (one mean per feature); dim=1 averages within each row. Same operation, opposite meanings, one integer apart.

dtype and device: the two silent attributes

Every tensor carries a dtype (float32 by default; int64 for class labels; smaller floats like float16/bfloat16 for speed — chapter 44 returns to this as quantization) and a device. The two crash classes they cause are famously blunt: an operation mixing CPU and GPU tensors errors immediately, and loss functions are picky about label dtypes. Both fixes are one call: x.to(device), y.long().

Where AI specifically gets this wrong

  • Shape comments that lie. Generated code annotates # (batch, dim) and then passes (dim,). Don't trust the comment — print(x.shape) is chapter 35's habit, and it carries straight over.
  • view/reshape roulette. Reshaping until the error stops without tracking which axis is which — the silent-transpose bug again, now with three or four axes to scramble.
  • Wrong dim=. Means-across-batch vs means-within-row. When a generated metric looks suspiciously constant, check the dim argument first.