详细的熊猫适用

Question

我在大熊猫系列上做了相对较重的申请。有没有什么方法可以获得一些打印反馈，说明每次调用函数时在函数内打印的距离是多少？

Answer 1

您可以使用跟踪器包装您的函数。下面两个例子，一个基于完成的迭代次数，一个基于总工作的百分比。

from pandas import Series

def progress_coroutine(print_on = 10000):
    print "Starting progress monitor"

    iterations = 0
    while True:
        yield
        iterations += 1
        if (iterations % print_on == 0):
            print "{} iterations done".format(iterations)

def percentage_coroutine(to_process, print_on_percent = 0.10):
    print "Starting progress percentage monitor"

    processed = 0
    count = 0
    print_count = to_process*print_on_percent
    while True:
        yield
        processed += 1
        count += 1
        if (count >= print_count):
            count = 0
            pct = (float(processed)/float(to_process))*100

            print "{}% finished".format(pct)

def trace_progress(func, progress = None):
    def callf(*args, **kwargs):
        if (progress is not None):
            progress.send(None)

        return func(*args, **kwargs)

    return callf

def my_func(i):
    return i * 2

data_series = Series(xrange(100000))
co1 = progress_coroutine()
co1.next()
co2 = percentage_coroutine(len(data_series))
co2.next()
data_series.apply(trace_progress(my_func, progress = co1))
data_series.apply(trace_progress(my_func, progress = co2))

Starting progress monitor
Starting progress percentage monitor
10000 iterations done
20000 iterations done
30000 iterations done
40000 iterations done
50000 iterations done
60000 iterations done
70000 iterations done
80000 iterations done
90000 iterations done
100000 iterations done
10.0% finished
20.0% finished
30.0% finished
40.0% finished
50.0% finished
60.0% finished
70.0% finished
80.0% finished
90.0% finished
100.0% finished

Answer 2

我对Python 3的上述代码进行了更改，以防有人想要使用。感谢@ Clockw3rk获得上述答案

from pandas import Series

def progress_coroutine(print_on = 10):
    print ("Starting progress monitor")

    iterations = 0
    while True:
        yield
        iterations += 1
        if (iterations % print_on == 0):
            print ("{} iterations done".format(iterations))

def percentage_coroutine(to_process, print_on_percent = 0.10):
    print ("Starting progress percentage monitor")

    processed = 0
    count = 0
    print_count = to_process*print_on_percent
    while True:
        yield
        processed += 1
        count += 1
        if (count >= print_count):
            count = 0
            pct = (float(processed)/float(to_process))*100

            print ("{}% finished".format(pct))

def trace_progress(func, progress = None):
    def callf(*args, **kwargs):
        if (progress is not None):
            progress.send(None)

        return func(*args, **kwargs)

    return callf

def my_func(i):
    return i ** 2

data_series = Series(list(range(100)))
co1 = progress_coroutine()
co2 = percentage_coroutine(len(data_series))
data_series.apply(trace_progress(my_func, progress = co1))
data_series.apply(trace_progress(my_func, progress = co2))

Starting progress monitor
10 iterations done
20 iterations done
30 iterations done
40 iterations done
50 iterations done
60 iterations done
70 iterations done
80 iterations done
90 iterations done
Starting progress percentage monitor
10.0% finished
20.0% finished
30.0% finished
40.0% finished
50.0% finished
60.0% finished
70.0% finished
80.0% finished
90.0% finished
Out[12]:
0        0
1        1
2        4
3        9
4       16
5       25
6       36
7       49
8       64
9       81
10     100
11     121
12     144
13     169
14     196
15     225
16     256
17     289
18     324
19     361
20     400
21     441
22     484
23     529
24     576
25     625
26     676
27     729
28     784
29     841
      ... 
70    4900
71    5041
72    5184
73    5329
74    5476
75    5625
76    5776
77    5929
78    6084
79    6241
80    6400
81    6561
82    6724
83    6889
84    7056
85    7225
86    7396
87    7569
88    7744
89    7921
90    8100
91    8281
92    8464
93    8649
94    8836
95    9025
96    9216
97    9409
98    9604
99    9801
Length: 100, dtype: int64

Answer 3

如果你需要一个简单的进度条：

from tqdm import tqdm
tqdm.pandas()
df.progress_apply(my_fun)

例如。申请翻译