promptdojo_

Array and Series shapes - the table before the table — step 1 of 7

Arrays and Series: the table before the table

Every ML dataset you will ever touch — training data, eval results, API logs — eventually becomes a table of rows and columns. The two libraries that own that shape in Python are numpy (raw numeric arrays) and pandas (labeled tables). Cursor reaches for them the moment you say "load this CSV" or "compute the average per user."

Browser note: this course's editor runs stdlib-only Python, so the runnable demos here use plain lists to teach the mental model. The fenced snippets show the real numpy/pandas calls you'll run on your own machine — the ideas transfer one-to-one.

The two core objects

  • numpy ndarray — a grid of values that all share one type (dtype). Its shape tells you the dimensions: (1000,) is a column of 1000 numbers, (1000, 12) is 1000 rows of 12 features.
  • pandas Series — a numpy array plus an index (labels for each position). A DataFrame is a dict of Series that share one index: that's your table.
import numpy as np
import pandas as pd

arr = np.array([10.0, 20.0, 30.0])     # ndarray, shape (3,), dtype float64
s   = pd.Series(arr, index=["a", "b", "c"])
df  = pd.DataFrame({"price": arr, "qty": [1, 2, 3]})
df.shape        # (3, 2) — 3 rows, 2 columns

The mental model: think in columns, not loops

The single biggest shift from the Python you know: you stop writing for loops over rows and start applying operations to whole columns. arr * 1.1 multiplies every element at once. `df["price"]

  • df["qty"]` multiplies two columns pairwise. This is called vectorization — the loop still happens, but inside numpy's compiled C code instead of your Python.

Run the editor: the loop version and the column version produce the same values. Numpy's win is that the column version is one line, and the compiled loop under it is much faster than a Python-level loop on big arrays.

Shape is the first thing you check

Almost every numpy/pandas bug is a shape bug. Before you compute anything, print the shape:

X.shape       # (rows, features) — is it what you expected?
len(df)       # row count
df.columns    # column names
df.dtypes     # one dtype per column

A model that expects (n_samples, n_features) and receives a flat (n_samples,) array will either crash or — worse — silently broadcast into nonsense.

Where AI specifically gets this wrong

  • Row loops over DataFrames. Cursor sometimes writes for i, row in df.iterrows(): for math that should be one vectorized column expression. It runs, slowly, and hides intent.
  • Ignoring dtype. A "numeric" column that loaded as strings (dtype: object) breaks math three steps later. Check df.dtypes right after loading — chapter lesson 03 drills this exact bug.
  • Trusting shape by vibes. Concatenating two arrays of shapes (100,) and (100, 1) does not do what the code pretends. Print shapes first.