Pandas and DataFrames

Meet the DataFrame

If NumPy is the engine, pandas is the steering wheel. A pandas DataFrame is a table with labelled rows and columns, like an Excel sheet you control with code. It is the tool you will use most in your career. Analysts at Zenith Bank use it to reconcile transactions, Jumia's data team uses tools like it to study orders, and Netflix data scientists use it to prototype recommendation features.

Each column in a DataFrame is a Series (a labelled NumPy array), and every column can have its own type.

Open in Google Colab

Follow along in a free cloud notebook. Nothing to install, and pandas is already available. Open a new notebook, then paste each code block from this lesson into its own cell and run it with Shift + Enter.

Launch a Colab notebook →

Step 1: Create and Load Data

In real work you load data from a file. So that this notebook runs anywhere, we first create a small CSV of mobile money transactions, then read it back exactly as you would a downloaded file.

import pandas as pd

csv_text = """txn_id,date,customer,city,channel,amount
T001,2025-01-03,Ada,Lagos,USSD,15000
T002,2025-01-03,Musa,Kano,App,42000
T003,2025-01-04,Ada,Lagos,App,8000
T004,2025-01-04,Ngozi,Enugu,POS,120000
T005,2025-01-05,Musa,Kano,USSD,5000
T006,2025-01-05,Tolu,Ibadan,App,67000
T007,2025-01-06,Ngozi,Enugu,App,23000
T008,2025-01-06,Ada,Lagos,POS,95000
"""
with open("transactions.csv", "w") as f:
    f.write(csv_text)

df = pd.read_csv("transactions.csv", parse_dates=["date"])
df.head()

read_csv is the function you will call thousands of times. It also reads straight from a URL, for example pd.read_csv("https://.../file.csv"), and has siblings: read_excel, read_json and read_sql.

Step 2: Inspect Before You Touch

Always get a feel for a new dataset first:

df.shape        # (8, 6) -> rows, columns
df.info()       # column types and non-null counts
df.describe()   # summary statistics for numeric columns
df["city"].value_counts()

Step 3: Select and Filter

df["amount"]                     # one column (a Series)
df[["customer", "amount"]]       # several columns (a DataFrame)

df[df["amount"] > 50000]         # rows where amount is over 50k
df[(df["city"] == "Lagos") & (df["channel"] == "App")]   # combine with &
df[df["channel"].isin(["USSD", "POS"])]

Notice the boolean mask from the NumPy lesson again. Use & (and), | (or) and ~ (not), and wrap each condition in brackets.

.loc selects by label and .iloc selects by position:

df.loc[df["amount"] > 50000, ["customer", "amount"]]
df.iloc[0:3]     # first three rows

Step 4: Create New Columns

df["amount_usd"] = (df["amount"] / 1550).round(2)
df["is_large"] = df["amount"] >= 50000
df["weekday"] = df["date"].dt.day_name()

Step 5: Group and Aggregate

groupby answers "per X, what is Y?" questions, and it is one of the most useful things in pandas.

df.groupby("city")["amount"].sum()

df.groupby("channel").agg(
    total=("amount", "sum"),
    average=("amount", "mean"),
    count=("txn_id", "count"),
).sort_values("total", ascending=False)

This is the same idea as SQL's GROUP BY or an Excel pivot table: split the rows into groups, apply a function to each group, and combine the results.

Step 6: Merge Tables

Real data lives in several tables. merge joins them on a shared key, just like a SQL JOIN.

customers = pd.DataFrame({
    "customer": ["Ada", "Musa", "Ngozi", "Tolu"],
    "bank": ["Access Bank", "GTBank", "Zenith Bank", "UBA"],
    "segment": ["Retail", "SME", "Retail", "SME"],
})

full = df.merge(customers, on="customer", how="left")
full.groupby("bank")["amount"].sum()

Step 7: Save Your Work

full.to_csv("transactions_enriched.csv", index=False)

In Colab, open the folder icon on the left to download the file.

Your Challenge

In your notebook, answer these three questions with pandas:

1. Which customer has the highest total spend?
2. What share of transactions go through USSD?
3. What is the average transaction amount for SME customers compared with Retail customers?

Use the browser lab below to check that you understand what each operation returns before you write the code.