Python for data
Lesson 1 of 9
1Python for data
The big picturepandas and NumPy are faster and stronger, but they rest on the same ideas. Once you see how to filter, group and sort a list of dictionaries in plain Python, every line of pandas afterwards makes sense.
What the picture shows: A small data pipeline. You will later write the same steps with one line of pandas each.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
Picture the café's sales ledger: each line a sale with the item, price and branch. You want the best-selling item in each branch. You would go through the lines, cross out the blank ones, add up per branch, then sort. Python does exactly that, a million times faster than you.
The core types: list (ordered), dict (key to value), set (no duplicates), tuple (fixed). A comprehension [f(x) for x in xs if cond] replaces most loops. csv.DictReader returns each row as a dictionary of strings, so convert the types yourself. collections.Counter and defaultdict for grouping, sorted(key=...) for ordering. Pure functions (no side effects) are easier to test and combine. And generators walk huge files without loading them whole.
Before You Start
🧭Start here
This path's foundation lesson.
The Core Idea
1A table = a list of dictionaries
We read CSV from a string (as if it were a file), convert the types, and drop broken rows in one line:
import csv
import io
raw = """item,price,branch
latte,14,riyadh
mocha,16,jeddah
latte,,riyadh
tea,6,riyadh
latte,14,jeddah
"""
rows = list(csv.DictReader(io.StringIO(raw)))
clean = [{**r, "price": int(r["price"])} for r in rows if r["price"]]
print(len(rows), "rows,", len(clean), "clean")
print(clean[0])csv reads everything as text. "14" + "6" gives "146", not 20. Convert numbers explicitly.2Grouping and sorting
defaultdict sums per key, Counter counts, and sorted with a key orders:
from collections import Counter, defaultdict
sales = [
{"item": "latte", "price": 14, "branch": "riyadh"},
{"item": "mocha", "price": 16, "branch": "jeddah"},
{"item": "tea", "price": 6, "branch": "riyadh"},
{"item": "latte", "price": 14, "branch": "jeddah"},
{"item": "latte", "price": 14, "branch": "riyadh"},
]
revenue = defaultdict(int)
for s in sales:
revenue[s["branch"]] += s["price"]
print("revenue:", dict(revenue))
popular = Counter(s["item"] for s in sales)
print("most sold:", popular.most_common(2))
by_price = sorted({s["item"]: s["price"] for s in sales}.items(), key=lambda kv: kv[1], reverse=True)
print("by price:", by_price)3JSON and generators
Most model APIs return JSON, and json.loads turns it into dictionaries and lists. For a huge file, a generator reads line by line (the JSONL format is common in training data):
import io
import json
jsonl = io.StringIO('{"text": "great coffee", "label": 1}\n'
'{"text": "cold and late", "label": 0}\n'
'{"text": "friendly staff", "label": 1}\n')
def records(stream):
for line in stream:
if line.strip():
yield json.loads(line)
positive = sum(r["label"] for r in records(jsonl))
print("positive examples:", positive)Worked example
Problem: a reviews file with 10,000 lines, some with no text and some duplicated. Steps: DictReader → filter if r["text"].strip() → dedupe with a set on the text → Counter on the rating. Result: 9,212 clean lines and the rating distribution, in 8 lines with no libraries. In the next lessons, the same thing with pandas.
Myth vs Reality
| Myth | Reality |
|---|---|
| pandas replaces core Python. | Every complex clean-up ends with a plain Python function you pass to pandas. |
| Loops are always slow. | For thousands of rows there is no real difference. For millions, NumPy and pandas are much faster. |
Test your understanding
What type does csv.DictReader read numbers as?
Everything is text, so convert it yourself.
Which tool counts each item's occurrences quickly?
Counter is a counting dictionary with most_common.
Why a generator for a huge file?
Memory holds one line at a time.
Quick review cards
Check Your Understanding
How do you represent a table in plain Python?
A list of dictionaries: each dictionary a row, keys being column names.
How do you remove duplicates while keeping order?
list(dict.fromkeys(items)).
When do you move to pandas?
When the data grows or you need many table operations. The understanding you built here comes with you.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Done with this lesson?
Sign in to save your progress.