pdb – How to use Python debugger

pdb, short for python debugger is a standard built-in module used to debug python code interactively. You can set breakpoints, see variable values, step inside routines, test run code, etc once you enter the pdb. This tutorial demonstrates the best practices of using the python debugger and the new breakpoint() feature.

Why use a debugger instead of print()?

Most of us start out debugging with print(). You sprinkle a few print statements, run the script, read the output, add a few more, run it again. It works, but it is slow, and you have to remember to remove the prints afterwards.

A debugger flips this around. You pause the program at the exact line you care about, look at every variable that exists at that moment, run any expression you like, and then step forward one line at a time. Nothing needs to be removed later. Since pdb ships with Python, there is nothing to install.

Setting a breakpoint with breakpoint()

A breakpoint is a spot in your code where you want the program to pause. Since Python 3.7, the simplest way to set one is the built-in breakpoint() function.

Say you have a function that computes an average and it is returning the wrong number.

def average(values):
    total = 0
    for v in values:
        total += v
    breakpoint()
    return total / len(values)


print(average([10, 20, 30]))

Run the script from the terminal as usual with python average.py. When Python reaches breakpoint(), it stops and hands you a (Pdb) prompt.

> average.py(6)average()
-> return total / len(values)
(Pdb)

The first line tells you the file, the line number and the function you are in. The arrow shows the line that is about to run next. You are now inside the debugger, and the program is frozen with all its variables intact.

Before Python 3.7 you had to write import pdb; pdb.set_trace() to get the same effect. You will still see that form in older code. Both work, but breakpoint() is shorter and can be switched off without editing the file (more on that below).

Inspecting variables

At the (Pdb) prompt, type the name of any variable to see its value. The p command prints an expression, and pp pretty prints it, which helps with long lists and dictionaries.

(Pdb) total
60
(Pdb) p values
[10, 20, 30]
(Pdb) p total / len(values)
20.0
(Pdb) pp {"total": total, "n": len(values)}
{'n': 3, 'total': 60}

You can run any Python expression here, call functions, even change a variable's value on the spot. To see every local variable at once, use p locals().

The a command (short for args) prints the arguments the current function was called with, and l (list) shows the source code around the current line so you never lose your bearings.

(Pdb) a
values = [10, 20, 30]
(Pdb) l
  1     def average(values):
  2         total = 0
  3         for v in values:
  4             total += v
  5         breakpoint()
  6  ->     return total / len(values)
  7
  8
  9     print(average([10, 20, 30]))

Stepping through code

Once you have looked around, you want to move. These are the commands you will use most.

The difference between n and s trips up most people at first. Use n when you trust the function being called and only want its result. Use s when the bug might be inside that function and you want to follow the call in.

Here is a small session that steps into a helper function.

def clean(text):
    text = text.strip()
    return text.lower()


def process(items):
    breakpoint()
    cleaned = [clean(x) for x in items]
    return cleaned


process(["  Apple", "BANANA "])

> process.py(8)process()
-> cleaned = [clean(x) for x in items]
(Pdb) s
--Call--
> process.py(1)clean()
-> def clean(text):
(Pdb) n
> process.py(2)clean()
-> text = text.strip()
(Pdb) n
> process.py(3)clean()
-> return text.lower()
(Pdb) p text
'Apple'
(Pdb) r
--Return--
> process.py(3)clean()->'apple'
-> return text.lower()
(Pdb) c

Pressing Enter on an empty line repeats the previous command, so after one n you can just keep hitting Enter to walk through the code line by line.

Conditional breakpoints

Suppose a loop runs ten thousand times and the bug only shows up on one particular iteration. Stopping on every pass would take forever. Wrap the breakpoint in an if so it fires only when the condition you care about is true.

for i, row in enumerate(rows):
    if row["amount"] < 0:
        breakpoint()
    total += row["amount"]

You can also set a conditional breakpoint from inside the debugger without touching the file. The b command takes a line number and an optional condition.

(Pdb) b 12, row["amount"] < 0
Breakpoint 1 at rows.py:12
(Pdb) c

The program now runs until line 12 is reached with a negative amount. Type b on its own to list all breakpoints, cl to clear them, and disable 1 to keep breakpoint 1 around but switch it off.

Running pdb on a script from the command line

You do not have to edit a file to debug it. Launch the script through the pdb module and it will pause on the very first line.

python -m pdb average.py

From there you can set breakpoints with b and continue with c. This is the way to go when you want to debug someone else's script without adding breakpoint() calls to it.

Post-mortem debugging: inspecting a crash after it happens

Often you do not know where to put a breakpoint until the program has already crashed with a traceback. Post-mortem debugging lets you open the debugger at the exact point where the exception was raised, with all the variables still in place.

The simplest way is the -m pdb flag again. When the script raises an unhandled exception, pdb catches it and drops you into the frame that failed.

python -m pdb average.py
(Pdb) c
Traceback (most recent call last):
  ...
ZeroDivisionError: division by zero
Uncaught exception. Entering post mortem debugging
> average.py(6)average()
-> return total / len(values)
(Pdb) p values
[]

Inside a longer program you can do the same thing with a try block and pdb.post_mortem().

import pdb
import sys

try:
    main()
except Exception:
    pdb.post_mortem(sys.exc_info()[2])

And if you are already at the interactive Python prompt when the error happens, pdb.pm() opens the debugger on the last exception.

>>> import pdb
>>> average([])
Traceback (most recent call last):
  ...
ZeroDivisionError: division by zero
>>> pdb.pm()
> average.py(6)average()
-> return total / len(values)
(Pdb)

Using pdb in Jupyter notebooks

breakpoint() works inside a notebook cell too. The (Pdb) prompt appears below the cell and accepts the same commands. Two shortcuts are worth knowing.

The %debug magic opens post-mortem debugging on the last error in the notebook. Run the cell that failed, then run %debug in the next cell, and you are inside the failing frame.

%debug
> <ipython-input-3>(2)average()
      1 def average(values):
----> 2     return sum(values) / len(values)
ipdb> values
[]

The %pdb on magic turns this on automatically, so every uncaught exception opens the debugger without you asking. Notebooks use ipdb, an extension of pdb with syntax highlighting and tab completion, but every command in this tutorial works the same way.

Turning breakpoints off without editing the code

One nice property of breakpoint() is that it reads the PYTHONBREAKPOINT environment variable. Set it to 0 and every breakpoint() call becomes a no-op, so a stray breakpoint that made it into a script cannot freeze a scheduled job.

PYTHONBREAKPOINT=0 python average.py

You can also point it at a different debugger, for example PYTHONBREAKPOINT=ipdb.set_trace, and breakpoint() will call that instead.

Common mistakes and how to avoid them

Shipping a breakpoint. The most common slip is leaving a breakpoint() in code that runs unattended. The program stops and waits for input that never comes. Search for breakpoint and set_trace before you commit, or set PYTHONBREAKPOINT=0 in production.

Typing a variable name that is also a pdb command. If you have a variable called n, l, c or s, typing its name runs the command instead of printing the variable. Use p n to print it. The same goes for a, b, r, u and d.

Using next when you meant step. If you press n over the line that calls the buggy function, the call runs to completion and you sail past the bug. Press s to go inside.

Trying to debug a comprehension one element at a time. List comprehensions run as a single line, so n executes the whole thing. Rewrite the comprehension as a plain for loop while you debug, then change it back.

Forgetting where you are in the stack. After a few s commands it is easy to end up three functions deep. Type w to print the stack, and u to move back up to the caller.

Summary

Put breakpoint() where you want the program to pause, run the script, and use p to look at variables, n and s to move, c to carry on and q to leave. Add a condition to the breakpoint when the bug only appears on one iteration, use python -m pdb or pdb.pm() to inspect a crash after the fact, and %debug in notebooks. Once these commands are in your fingers, you will reach for the debugger long before you reach for print().