熊猫：有条件地聚合连续行

Question

我有一个带有连续索引的数据框（每个日历日的日期）和一个不包含每个日期的参考向量（仅限工作日）。

我想将数据帧重新索引到参考向量中的日期，其中缺失的数据在缺失日期部分之前聚合到最新的条目（即周末数据应汇总到一起）上周五）。

目前我已经通过循环反转索引并收集周末数据，然后在循环中添加它来实现这一点。 我在问是否有更高效的“阵列方式”来做这件事。

import pandas as pd
import numpy as np
df = pd.DataFrame({'x': np.arange(10), 'y': np.arange(10)**2},
                  index=pd.date_range(start="2018-01-01", periods=10))
print(df)
ref_dates = pd.date_range(start="2018-01-01", periods=10)
ref_dates = ref_dates[:5].append(ref_dates[7:])  # omit 2018-01-06 and -07

# inefficient approach by reverse-traversing the dates, collecting the data
# and aggregating it together with the first date that's in ref_dates
df.sort_index(ascending=False, inplace=True)
collector = []
for dt in df.index:
    if collector and dt in ref_dates:
        # data from previous iteration was collected -> aggregate it and reset collector
        # first append also the current data
        collector.append(df.loc[dt, :].values)
        collector = np.array(collector)

        # applying aggregation function, here sum as example
        aggregates = np.sum(collector, axis=0)

        # setting the new data
        df.loc[dt,:] = aggregates

        # reset collector
        collector = []

    if dt not in ref_dates:
        collector.append(df.loc[dt, :].values)

df = df.reindex(ref_dates)
print(df)

给出输出（第一个：源数据帧，第二个：目标数据帧）

            x   y
2018-01-01  0   0
2018-01-02  1   1
2018-01-03  2   4
2018-01-04  3   9
2018-01-05  4  16
2018-01-06  5  25
2018-01-07  6  36
2018-01-08  7  49
2018-01-09  8  64
2018-01-10  9  81
             x   y
2018-01-01   0   0
2018-01-02   1   1
2018-01-03   2   4
2018-01-04   3   9
2018-01-05  15  77   # contains the sum of Jan 5th, 6th and 7th
2018-01-08   7  49 
2018-01-09   8  64
2018-01-10   9  81

Answer 1

仍然有一个列表理解循环，但有效。

import pandas as pd
import numpy as np

# Create dataframe which contains all days
df = pd.DataFrame({'x': np.arange(10), 'y': np.arange(10)**2},
                  index=pd.date_range(start="2018-01-01", periods=10))

# create second dataframe which only contains week-days or whatever dates you need.
ref_dates = [x for x in df.index if x.weekday() < 5]

# Set the index of df to a forward filled version of the ref days
df.index = pd.Series([x if x in ref_dates else float('nan') for x in df.index]).fillna(method='ffill')

# Group by unique dates and sum
df = df.groupby(level=0).sum()

print(df)