我如何使用line_profiler(来自Robert Kern)?

时间:2014-05-27 09:00:48

标签: python python-2.7 profiling line-profiler

我尝试使用line_profiler模块获取Python文件的逐行配置文件。这是我到目前为止所做的:

1)使用.exe文件(我在WinXP和Win7上)从pypi安装了line_profiler。只需单击安装向导即可。

2)写了一小段代码(类似于另一个回答的问题here中提到的内容)。

from line_profiler import LineProfiler
def do_stuff(numbers):
    print numbers

numbers = 2
profile = LineProfiler(do_stuff(numbers))
profile.print_stats()

3)运行IDLE / PyScripter中的代码。我只得到了时间。

Timer unit: 4.17188e-10 s

如何在我执行的代码上获得完整的逐行配置文件?我从未使用任何高级Python功能,如装饰器,所以我很难理解如何使用herehere等几个帖子提供的指南。

4 个答案:

答案 0 :(得分:28)

这个答案是我的答案here的副本,用于了解如何从Python脚本中获取line_profiler统计信息(不使用命令行中的kernprof或必须添加{{1}装饰函数和类方法)。我对类似@profile个问题的所有答案(我已经看过)仅描述使用line_profiler

kernprof测试用例(在GitHub上找到)有一个如何从Python脚本中生成配置文件数据的示例。您必须包装要分析的函数,然后调用包装器传递任何所需的函数参数。

line_profiler

输出:

from line_profiler import LineProfiler
import random

def do_stuff(numbers):
    s = sum(numbers)
    l = [numbers[i]/43 for i in range(len(numbers))]
    m = ['hello'+str(numbers[i]) for i in range(len(numbers))]

numbers = [random.randint(1,100) for i in range(1000)]
lp = LineProfiler()
lp_wrapper = lp(do_stuff)
lp_wrapper(numbers)
lp.print_stats()

向个人资料添加附加功能

此外,您还可以添加要分析的其他功能。例如,如果您有第二个调用函数并且只包装了调用函数,那么您只能看到来自调用的配置文件结果功能。

Timer unit: 1e-06 s

Total time: 0.000649 s
File: <ipython-input-2-2e060b054fea>
Function: do_stuff at line 4

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     4                                           def do_stuff(numbers):
     5         1           10     10.0      1.5      s = sum(numbers)
     6         1          186    186.0     28.7      l = [numbers[i]/43 for i in range(len(numbers))]
     7         1          453    453.0     69.8      m = ['hello'+str(numbers[i]) for i in range(len(numbers))]

以上只会为调用函数生成以下配置文件输出:

from line_profiler import LineProfiler
import random

def do_other_stuff(numbers):
    s = sum(numbers)

def do_stuff(numbers):
    do_other_stuff(numbers)
    l = [numbers[i]/43 for i in range(len(numbers))]
    m = ['hello'+str(numbers[i]) for i in range(len(numbers))]

numbers = [random.randint(1,100) for i in range(1000)]
lp = LineProfiler()
lp_wrapper = lp(do_stuff)
lp_wrapper(numbers)
lp.print_stats()

在这种情况下,您可以将其他名为的函数添加到配置文件中,如下所示:

Timer unit: 1e-06 s

Total time: 0.000773 s
File: <ipython-input-3-ec0394d0a501>
Function: do_stuff at line 7

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     7                                           def do_stuff(numbers):
     8         1           11     11.0      1.4      do_other_stuff(numbers)
     9         1          236    236.0     30.5      l = [numbers[i]/43 for i in range(len(numbers))]
    10         1          526    526.0     68.0      m = ['hello'+str(numbers[i]) for i in range(len(numbers))]

输出:

from line_profiler import LineProfiler
import random

def do_other_stuff(numbers):
    s = sum(numbers)

def do_stuff(numbers):
    do_other_stuff(numbers)
    l = [numbers[i]/43 for i in range(len(numbers))]
    m = ['hello'+str(numbers[i]) for i in range(len(numbers))]

numbers = [random.randint(1,100) for i in range(1000)]
lp = LineProfiler()
lp.add_function(do_other_stuff)   # add additional function to profile
lp_wrapper = lp(do_stuff)
lp_wrapper(numbers)
lp.print_stats()

注意:以这种方式向配置文件添加功能不需要更改配置文件代码(即,无需添加Timer unit: 1e-06 s Total time: 9e-06 s File: <ipython-input-4-dae73707787c> Function: do_other_stuff at line 4 Line # Hits Time Per Hit % Time Line Contents ============================================================== 4 def do_other_stuff(numbers): 5 1 9 9.0 100.0 s = sum(numbers) Total time: 0.000694 s File: <ipython-input-4-dae73707787c> Function: do_stuff at line 7 Line # Hits Time Per Hit % Time Line Contents ============================================================== 7 def do_stuff(numbers): 8 1 12 12.0 1.7 do_other_stuff(numbers) 9 1 208 208.0 30.0 l = [numbers[i]/43 for i in range(len(numbers))] 10 1 474 474.0 68.3 m = ['hello'+str(numbers[i]) for i in range(len(numbers))] 装饰器)。

答案 1 :(得分:12)

请关注第一个link中的Dan Riti示例,但请使用您的代码。安装line_profiler模块后,您只需要在每个要逐行分析的函数之前添加@profile装饰器,并确保每个函数至少在其他位置调用一次。代码 - 所以你的琐碎的示例代码将是这样的:

example.py档案:

@profile
def do_stuff(numbers):
    print numbers

numbers = 2
do_stuff(numbers)

完成后,通过kernprof.py目录中安装的C:\Python27\Scripts 运行脚本。这是在Windows 7命令行会话中执行此操作的实际输出(不是非常有趣):

> python "C:\Python27\Scripts\kernprof.py" -l -v example.py
2
Wrote profile results to example.py.lprof
Timer unit: 3.2079e-07 s

File: example.py
Function: do_stuff at line 2
Total time: 0.00185256 s

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     1                                           @profile
     2                                           def do_stuff(numbers):
     3         1         5775   5775.0    100.0      print numbers

您可能需要调整最后一步 - 使用kernprof.py而不是直接通过Python解释器运行测试脚本 - 以便在IDLE或PyScripter中执行等效操作。

<强> 更新

似乎在line_profiler v1.0中,kernprof实用程序作为可执行文件分发,而不是像我上面编写时那样的.py脚本文件。这意味着现在需要使用以下命令从命令行调用它:

> "C:\Python27\Scripts\kernprof.exe" -l -v example.py

答案 2 :(得分:2)

如果你使用的是 PyCharm,你也可以看看 https://plugins.jetbrains.com/plugin/16536-line-profiler

这是我创建的一个插件,允许您将线分析器结果加载到 PyCharm 编辑器中并将其可视化。

答案 3 :(得分:1)

加载 line_profiler 和 numpy

%load_ext line_profiler
import numpy as np

定义一个函数例如:

def take_sqr(array):
    sqr_ar = [np.sqrt(x) for x in array]
    return sqr_ar

使用line_profiler计算时间如下:

%lprun -f take_sqr take_sqr([1,2,3])

输出如下所示:

Timer unit: 1e-06 s
 
Total time: 6e-05 s File: <ipython-input-5-e50c1b05a473> Function:
take_sqr at line 1

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     1                                           def take_sqr(array):
     2         4         59.0     14.8     98.3      sqr_ar = [np.sqrt(x) for x in array]
     3         1          1.0      1.0      1.7      return sqr_ar