Picture by Writer
As a software program developer, you’ll have doubtless heard the quote “Untimely optimization is the foundation of all evil”—greater than as soon as—in your profession. Whereas optimization might not be tremendous useful (or completely needed) for small initiatives, profiling is usually useful.
After you’ve completed coding a module, it’s follow to profile your code to measure how lengthy every of the sections takes to execute. This will help establish code smells and information optimizations to enhance code high quality. So at all times profile your code earlier than optimizing!
To take the primary steps, this information will enable you get began with profiling in Python—utilizing the built-in timeit and cProfile modules. You’ll study to make use of each the command-line interface and the equal callables inside Python scripts.
The timeit module is a part of the Python normal library and provides just a few comfort capabilities that can be utilized to time brief snippets of code.
Let’s take a easy instance of reversing a Python listing. We’ll measure the execution instances of acquiring a reversed copy of the listing utilizing:
- the
reversed()operate, and - listing slicing.
>>> nums=[6,9,2,3,7]
>>> listing(reversed(nums))
[7, 3, 2, 9, 6]
>>> nums[::-1]
[7, 3, 2, 9, 6]
Operating timeit on the Command Line
You may run timeit on the command line utilizing the syntax:
$ python -m timeit -s 'setup-code' -n 'quantity' -r 'repeat' 'stmt'
You’re required to offer the assertion stmt whose execution time is to be measured.
You may specify the setup code when wanted—utilizing the brief choice -s or the lengthy choice –setup. The setup code can be run solely as soon as.
The quantity of instances to run the assertion: brief choice -n or the lengthy choice –number is non-obligatory. And the variety of instances to repeat this cycle: brief choice -r or the lengthy choice –repeat is non-obligatory, too.
Let’s see the above in motion for our instance:
Right here creating the listing is the setup code and reversing the listing is the assertion to be timed:
$ python -m timeit -s 'nums=[6,9,2,3,7]' 'listing(reversed(nums))'
500000 loops, better of 5: 695 nsec per loop
While you don’t specify values for repeat, the default worth of 5 is used. And while you don’t specify quantity, the code is run as many instances as wanted in order to succeed in a complete time of at the least 0.2 seconds.
This instance explicitly units the variety of instances to execute the assertion:
$ python -m timeit -s 'nums=[6,9,2,3,7]' -n 100Bu000 'listing(reversed(nums))'
100000 loops, better of 5: 540 nsec per loop
The default worth of repeat is 5, however we will set it to any appropriate worth:
$ python3 -m timeit -s 'nums=[6,9,2,3,7]' -r 3 'listing(reversed(nums))'
500000 loops, greatest of three: 663 nsec per loop
Let’s additionally time the listing slicing strategy:
$ python3 -m timeit -s 'nums=[6,9,2,3,7]' 'nums[::-1]'
1000000 loops, better of 5: 142 nsec per loop
The listing slicing strategy appears to be quicker (all examples are in Python 3.10 on Ubuntu 22.04).
Operating timeit in a Python Script
Right here’s the equal of working timeit contained in the Python script:
import timeit
setup = 'nums=[9,2,3,7,6]'
quantity = 100000
stmt1 = 'listing(reversed(nums))'
stmt2 = 'nums[::-1]'
t1 = timeit.timeit(setup=setup,stmt=stmt1,quantity=quantity)
t2 = timeit.timeit(setup=setup,stmt=stmt2,quantity=quantity)
print(f"Utilizing reversed() fn.: {t1}")
print(f"Utilizing listing slicing: {t2}")
The timeit() callable returns the execution time of stmt for quantity of instances. Discover that we will explicitly point out the variety of instances to run, or make quantity take the default worth of 1000000.
Output >>
Utilizing reversed() fn.: 0.08982690000000002
Utilizing listing slicing: 0.015550800000000004
This runs the assertion—with out repeating the timer operate—for the desired quantity of instances and returns the execution time. Additionally it is fairly frequent to make use of time.repeat() and take the minimal time as proven:
import timeit
setup = 'nums=[9,2,3,7,6]'
quantity = 100000
stmt1 = 'listing(reversed(nums))'
stmt2 = 'nums[::-1]'
t1 = min(timeit.repeat(setup=setup,stmt=stmt1,quantity=quantity))
t2 = min(timeit.repeat(setup=setup,stmt=stmt2,quantity=quantity))
print(f"Utilizing reversed() fn.: {t1}")
print(f"Utilizing listing slicing: {t2}")
This can repeat the method of working the code quantity of instances repeat variety of instances and returns the minimal execution time. Right here we’ve got 5 repetitions of 100000 instances every.
Output >>
Utilizing reversed() fn.: 0.055375300000000016
Utilizing listing slicing: 0.015101400000000043
We’ve got seen how timeit can be utilized to measure the execution instances of brief code snippets. Nonetheless, in follow, it is extra useful to profile a complete Python script.
This can give us the execution instances of all of the capabilities and technique calls—together with built-in capabilities and strategies. So we will get a greater concept of the costlier operate calls and establish alternatives for optimization. For instance: there is likely to be an API name that is too sluggish. Or a operate could have a loop that may be changed by a extra Pythonic comprehension expression.
Let’s discover ways to profile Python scripts utilizing the cProfile module (additionally a part of the Python normal library).
Think about the next Python script:
# important.py
import time
def func(num):
for i in vary(num):
print(i)
def another_func(num):
time.sleep(num)
print(f"Slept for {num} seconds")
def useful_func(nums, goal):
if goal in nums:
return nums.index(goal)
if __name__ == "__main__":
func(1000)
another_func(20)
useful_func([2, 8, 12, 4], 12)
Right here we’ve got three capabilities:
func()that loops via a spread of numbers and prints them out.one other func()that incorporates a name to thesleep()operate.useful_func()that returns the index of a goal quantity in listing (if the goal is current within the listing).
The above-listed capabilities can be known as every time you run the principle.py script.
Operating cProfile on the Command Line
Run cProfile on the command line utilizing:
Right here we’ve named the file important.py:
Operating this could provide the following output:
Output >>
0
...
999
Slept for 20 seconds
And the next profile:
Right here, ncalls refers back to the variety of calls to the operate and percall refers back to the time per operate name. If the worth of ncalls is larger than one, then percall is the common time throughout all calls.
The execution time of script is dominated by another_func that makes use of the built-in sleep operate name (sleeps for 20 seconds). We see that print operate calls are fairly costly too.
Utilizing cProfile within the Python Script
Whereas working cProfile on the command line works tremendous, you can even add the profiling performance to the Python script. You need to use cProfile coupled with the pstats module for profiling and accessing statistics.
As a greatest follow to deal with useful resource setup and teardown higher, use the with assertion and create a profile object that’s used as a context supervisor:
# important.py
import pstats
import time
import cProfile
def func(num):
for i in vary(num):
print(i)
def another_func(num):
time.sleep(num)
print(f"Slept for {num} seconds")
def useful_func(nums, goal):
if goal in nums:
return nums.index(goal)
if __name__ == "__main__":
with cProfile.Profile() as profile:
func(1000)
another_func(20)
useful_func([2, 8, 12, 4], 12)
profile_result = pstats.Stats(profile)
profile_result.print_stats()
Let’s take a better take a look at the output profile generated:
While you’re profiling a big script, it’ll be useful to type the outcomes by execution time. To take action you may name sort_stats on the profile object and kind primarily based on the execution time:
...
if __name__ == "__main__":
with cProfile.Profile() as profile:
func(1000)
another_func(20)
useful_func([2, 8, 12, 4], 12)
profile_result = pstats.Stats(profile)
profile_result.sort_stats(pstats.SortKey.TIME)
profile_result.print_stats()
While you now run the script, it is best to be capable to see the outcomes sorted by time:
I hope this information helps you get began with profiling in Python. All the time keep in mind, optimizations ought to by no means come at the price of readability. For those who’re all in favour of studying about different profilers, together with third-party Python packages, try this article on Python profilers.
Bala Priya C is a developer and technical author from India. She likes working on the intersection of math, programming, information science, and content material creation. Her areas of curiosity and experience embrace DevOps, information science, and pure language processing. She enjoys studying, writing, coding, and occasional! At present, she’s engaged on studying and sharing her data with the developer group by authoring tutorials, how-to guides, opinion items, and extra.