pandas for Beginners: The Ten Operations You Will Actually Use

Quick Answer

Learn read_csv, head, shape, boolean filtering, groupby, sort_values, apply, isna and fillna. Those cover most real work. The two things that confuse beginners are that filtering returns a copy rather than modifying in place, and that mean() silently skips missing values.

A DataFrame is a table with names

A DataFrame is rows and columns, where columns have names and types. In real work you load one from a file with pd.read_csv("marks.csv"); here it is built inline so you can run everything as-is.

import pandas as pd

df = pd.DataFrame({
    "name":   ["Asha", "Ravi", "Meera", "Zoya", "Iqbal"],
    "stream": ["Science", "Commerce", "Science", "Commerce", "Science"],
    "marks":  [91, 68, 96, 78, 84],
})

print(df.head(3))
print(df.shape)      # (5, 3)

head(n) shows the first n rows and is the first thing to run on any new dataset. shape gives (rows, columns). Together they answer "what am I actually looking at", which is the question beginners skip and then debug for an hour.

Filtering rows

You select rows by writing a condition, which produces a column of True/False that pandas uses as a mask:

print(df[df.marks > 80])

  name   stream  marks
  Asha  Science     91
 Meera  Science     96
 Iqbal  Science     84

Combine conditions with & and |, and each condition needs its own brackets: df[(df.marks > 80) & (df.stream == "Science")]. Python's and and or do not work here and will raise an error about ambiguous truth values — a message that makes no sense until you know this rule.

The important thing to understand: this returns a new DataFrame. The original df is unchanged. Beginners often filter, print the right answer, then wonder why the next line still shows everything.

groupby: the one that makes pandas worth learning

Split rows into groups, compute something per group. This single operation replaces a surprising amount of loop-and-dictionary code.

print(df.groupby("stream")["marks"].mean())

stream
Commerce    73.000000
Science     90.333333

Read it as three steps: group by stream, take the marks column, average it. Swap mean() for sum(), count(), max() or min() as needed. If you have written SQL, this is GROUP BY and it behaves the same way — see SQL vs NoSQL for where that thinking transfers.

Sorting and computing new columns

print(df.sort_values("marks", ascending=False).head(2))

  name   stream  marks
 Meera  Science     96
  Asha  Science     91

To derive a new column, apply runs a function over every value:

df["grade"] = df["marks"].apply(
    lambda m: "A" if m >= 90 else "B" if m >= 75 else "C")
print(df)

  name   stream  marks grade
  Asha  Science     91     A
  Ravi Commerce     68     C
 Meera  Science     96     A
  Zoya Commerce     78     B
 Iqbal  Science     84     B

For simple arithmetic you do not need apply at all — df["marks"] * 1.05 operates on the whole column at once and is considerably faster. Reach for apply when the logic genuinely needs a function.

Missing values, and why your average is wrong

Real data has holes. pandas represents them as NaN, and its behaviour around them is the single most common source of quietly wrong results.

missing = pd.DataFrame({"name": ["A", "B", "C"],
                        "marks": [90, None, 70]})

print(missing["marks"].isna().sum())   # 1
print(missing["marks"].mean())         # 80.0

That mean is 80.0 — the average of 90 and 70. The missing row was skipped, not counted as zero. That is usually what you want, but it means the denominator changed silently. If a report is subtly off, this is the first thing to check.

Your options are fillna(0) or fillna(df["marks"].mean()) to substitute a value, or dropna() to remove the rows. Which is correct depends entirely on what the missing value means — a student who was absent is not the same as a student who scored zero, and choosing wrong here changes your conclusion.

Where to go next

Those operations cover most day-to-day analysis. The natural next steps are merge for joining two DataFrames, pivot_table for cross-tabulation, and plotting with matplotlib.

But the far more useful next step is applying it to data you care about rather than a tutorial dataset. Download something real — your own expenses, a public dataset, your college's results — and answer one specific question about it. That produces something you can actually discuss, unlike a notebook that repeats a tutorial. Our pandas data analysis project and retail sales forecasting builds are structured that way.

Frequently Asked Questions