Profile a Python script with cProfile

To profile Python code by function, start with python3 -m cProfile -s cumulative myscript.py and inspect cumulative time. Use the results to choose where to investigate, then measure a representative workload before deciding whether a change helped.

Profile from the command line

python3 -m cProfile -s cumulative myscript.py

The report includes:

High cumulative time identifies an expensive call path. High internal time can identify a function doing substantial work itself. These are function-level measurements, not a breakdown of every source line.

Profile a specific call

import cProfile

def my_function():
    return sum(value * value for value in range(10_000))

cProfile.run("my_function()")

For code that already holds a function reference and arguments, cProfile.Profile().runcall(my_function) avoids expressing the call as a string.

Save and inspect statistics

python3 -m cProfile -o profile_stats myscript.py

Then inspect the file:

import pstats

stats = pstats.Stats("profile_stats")
stats.sort_stats("cumulative").print_stats(20)
stats.sort_stats("time").print_stats(20)

The first listing ranks cumulative time; the second ranks internal time. Keep workload, interpreter, and environment comparable when comparing runs. A profile is not a machine-independent performance score.

Inspect individual lines

The third-party line_profiler package can measure selected functions line by line:

python3 -m pip install line_profiler
kernprof -l -v myscript.py

In the script, mark the target function with @profile for the kernprof workflow. That decorator is supplied by the tool; it is not a built-in decorator available in an ordinary Python run. Consult the installed version's documentation if you prefer an explicit import workflow.

When the problem is memory rather than execution time

cProfile and line_profiler measure execution time, not memory use. For allocations tracked by Python, start the standard-library tracemalloc before the work you want to inspect, then compare snapshots:

import tracemalloc

tracemalloc.start()
before = tracemalloc.take_snapshot()
payloads = [bytearray(1024) for _ in range(1_000)]
after = tracemalloc.take_snapshot()

for change in after.compare_to(before, "lineno")[:5]:
    print(change)
tracemalloc.stop()

Replace the allocation with the operation under investigation. This reports changes in traced allocations still present at each snapshot, not every temporary allocation or the process's full resident memory. It is useful for locating growth in Python allocations; native allocations and total process memory may need a separate measurement.

The third-party memory_profiler package offers process and line-by-line memory measurements, but its maintainers state that it is no longer actively maintained. Its default psutil backend measures RSS, which differs from Python allocation tracking. Check the backend, interpreter compatibility, and maintenance needs before adopting it; the tool's name alone does not define what its memory numbers include.

FAQ

How much overhead does profiling add? It depends on the workload and profiler. There is no reliable universal 10–30% overhead figure. Treat instrumented timing as diagnostic evidence and validate important improvements without the profiler.

When should I use timeit? Use it to compare small pieces of code through repeated executions. cProfile helps locate expensive functions in a larger run.

Can I visualize the output? Tools such as SnakeViz and gprof2dot consume profiling statistics. They make call paths easier to explore, but do not eliminate measurement overhead or make the workload representative.

References

• Python profiling tools
• Line Profiler documentation