# df
date value
0 2018-01-22 01:01:53.192824 1
1 2018-01-22 01:01:55.042070 2
2 2018-01-22 01:01:56.264234 3
3 2018-01-22 01:01:57.697656 2
4 2018-01-22 01:01:57.831543 2
5 2018-01-22 01:02:00.258684 1
6 2018-01-22 01:02:00.259691 3
7 2018-01-22 01:02:00.260698 2
8 2018-01-22 01:02:00.261683 1
9 2018-01-22 01:02:00.333109 2
我的目标是创建一个字典，其中包含与每分钟对应的键以及每次计算最后3个值的结果。
如果最后3个值的序列不是连续上升或下降，则计算累计为1.
简单地说，如果最后3个值是↘↗或↗↘，则将1加到其键上。
例如，在2018-01-22 01：01：56.264234，最后3个值是1,2,3并且它们正在增加，您不添加1.
但是在2018-01-22 01：01：57.697656，最后3个值是2,3,2，它们就像↗↘，你加1。
上面的数据框将生成如下字典：
dic_result = { np.datetime('2018-01-22 01:01'): 1, # [2, 3, 2]
np.datetime('2018-01-22 01:02'): 3 } # [2, 1, 3], [1, 3, 2], [2, 1, 2]
这是我编程完成这项工作并且工作正常，但如果数据框很大则需要花费太多时间。 我希望知道如何改进这些代码并获得更好的性能，例如，使用numpy数组或更好的算法。
# I used deque to store last 3 values
deq_3_trs = deque(maxlen=3)
dic_result = {}
for i in range( len(df) ):
date = df.ix[i]['date']
date_min = np.datetime64(date, 'm')
value = df.ix[i]['value']
deq_3_trs.append(value)
if (date_min not in dic_result) and (len(deq_3_trs) == 3):
dic_result[date_min] = 0
# check the deque if the values are like either ↘↗ or ↗↘
if (deq_3_trs[0] > deq_3_trs[1] < deq_3_trs[2]) or (deq_3_trs[0] < deq_3_trs[1] > deq_3_trs[2]):
dic_result[date_min] += 1
elif (date_min in dic_result) and (len(deq_3_trs) == 3):
# check the deque if the values are like either ↘↗ or ↗↘
if (deq_3_trs[0] > deq_3_trs[1] < deq_3_trs[2]) or (deq_3_trs[0] < deq_3_trs[1] > deq_3_trs[2]):
dic_result[date_min] += 1
在情况i == 0，i == 2和i == 3
的情况下流动图表0 2018-01-22 01:01:53.192824 1
1 2018-01-22 01:01:55.042070 2
2 2018-01-22 01:01:56.264234 3
3 2018-01-22 01:01:57.697656 2
4 2018-01-22 01:01:57.831543 2
5 2018-01-22 01:02:00.258684 1
6 2018-01-22 01:02:00.259691 3
7 2018-01-22 01:02:00.260698 2
8 2018-01-22 01:02:00.261683 1
9 2018-01-22 01:02:00.333109 2
If i == 0 in the FOR loop,
date == 2018-01-22 01:01:53.192824
date_min == numpy.datetime64('2018-01-22T01:01')
value == 1
deq_3_trs == deque([1], maxlen=3)
Since len(deq_3_trs) != 3, the FOR loop ends now.
If i == 2 in the FOR loop,
date == 2018-01-22 01:01:56.264234
date_min == numpy.datetime64('2018-01-22T01:01')
value == 3
deq_3_trs == deque([1, 2, 3], maxlen=3)
Since len(deq_3_trs) == 3 and the dictionary dic_result has no key as 'numpy.datetime64('2018-01-22T01:01')',
it creats the key and defaults it to 0. dic_result == { 'numpy.datetime64('2018-01-22T01:01')':0 }
The series of values in the deque is not like ↘↗ or ↗↘, the FOR loop ends now.
If i == 3 in the FOR loop,
date == 2018-01-22 01:01:57.697656
date_min == numpy.datetime64('2018-01-22T01:01')
value == 2
deq_3_trs == deque([2, 3, 2], maxlen=3)
Since len(deq_3_trs) == 3 and the dictionary dic_result has the 'numpy.datetime64('2018-01-22T01:01')' and
the series of values in the deque is like ↗↘, it adds 1 to the key. dic_result == { 'numpy.datetime64('2018-01-22T01:01')':1 }
答案 0 :(得分：3)
这可能有用。迭代行似乎不太理想，但您可能会从collections中受益。
import pandas as pd
from collections import defaultdict
df = pd.DataFrame([['2018-01-22 01:01:53.192824', 1], ['2018-01-22 01:01:55.042070', 2],
['2018-01-22 01:01:56.264234', 3], ['2018-01-22 01:01:57.697656', 2],
['2018-01-22 01:01:57.831543', 2], ['2018-01-22 01:02:00.258684', 1],
['2018-01-22 01:02:00.259691', 3], ['2018-01-22 01:02:00.260698', 2],
['2018-01-22 01:02:00.261683', 1], ['2018-01-22 01:02:00.333109', 2]],
columns=['date', 'value'])
df['date'] = pd.to_datetime(df['date'])
df = df.set_index('date')
df.index = df.index.map(lambda x: x.replace(second=0).replace(microsecond=0))
result = defaultdict(list)
def not_noninc_or_nondec(L):
return not (all(x>=y for x, y in zip(L, L[1:])) or all(x<=y for x, y in zip(L, L[1:])))
for i, (idx, row) in enumerate(df.iterrows()):
if i >= 2:
result[idx].append(not_noninc_or_nondec(df['value'][i-2:i+1].tolist()))
result_count = {k: sum(v) for k, v in result.items()}
# {Timestamp('2018-01-22 01:01:00'): 1, Timestamp('2018-01-22 01:02:00'): 3}
答案 1 :(得分：1)
@jp_data_analysis您的解决方案看起来非常好。另外，我建议not_noninc_or_nondec()的以下代码更容易理解：
def not_noninc_or_nondec(L):
return (L[0]-L[1])*(L[1]-L[2]) < 0