DataFrame selection and cleaning without guessing — step 1 of 7

Selecting rows: .loc, .iloc, and boolean masks

Half of real pandas work is just "give me the rows I care about." Pandas has two selection systems, and mixing them up is the classic beginner bug.

Label vs position

• df.loc[...] selects by label — the index values and column names. df.loc["marcus", "score"].
• df.iloc[...] selects by integer position — row 0, row 1. df.iloc[1, 0].

Run the editor. After a sort, positions shuffle but labels stick to their rows. That's the whole distinction, and it's why position-based code breaks the moment someone sorts or filters upstream of it.

One more sharp edge worth memorizing: .loc slices are inclusive of the end label — df.loc["a":"c"] includes "c" — while .iloc slices follow normal Python rules and exclude the end position. Same-looking code, off-by-one different results.

Boolean masks: the filter workhorse

The pattern AI writes most in pandas:

active = df[df["status"] == "active"]
big    = df[(df["amount"] > 100) & (df["region"] == "EU")]

The expression inside the brackets is a mask — a column of True/False, one per row. Two gotchas:

• Use & and |, not Python's and/or — those raise a ValueError on a whole column. And keep the parentheses around each condition: & binds tighter than comparisons, so without them Python parses the expression wrong before pandas even sees it.
• Don't write through a filter. Under modern pandas (3.x copy-on-write), assigning into a chained selection never updates the original — it raises ChainedAssignmentError. The safe write pattern is one step: df.loc[mask, "col"] = value.

Basic cleaning moves

df = df.drop_duplicates()
df = df.rename(columns={"Amt": "amount"})
df["amount"] = df["amount"].astype(float)
df = df.sort_values("created_at")

Each returns a new frame (pandas is mostly non-mutating by default — same philosophy as chapter 07 taught for lists: prefer new objects over in-place surprises).

Where AI specifically gets this wrong

• iloc where it meant loc. Generated code hardcodes row positions that were only correct for the demo ordering.
• and/or on columns. Instant ValueError about ambiguous truth values. The fix is &/| with parentheses.

And the classic: chained indexing. df[df.x > 0]["y"] = 1 looks fine and never writes — pandas 3.x raises ChainedAssignmentError, while older pandas sometimes just silently didn't write. One-step .loc[mask, "y"] = 1 is the fix either way.