Question

我正在尝试编写一个连续时间序列的函数，并返回描述数据中任何缺失空白的数据结构（例如，带有'start'和'end'列的DF）。对于时间序列来说这似乎是一个相当普遍的问题，但是尽管乱搞了groupby，diff等等 - 并且正在探索SO - 我还没有能够提出比下面更好的东西。

对我来说，优先使用矢量化操作以保持高效。使用矢量化操作必须有一个更明显的解决方案 - 不是吗？感谢任何帮助，伙计们。

import pandas as pd


def get_gaps(series):
    """
    @param series: a continuous time series of data with the index's freq set
    @return: a series where the index is the start of gaps, and the values are
         the ends
    """
    missing = series.isnull()
    different_from_last = missing.diff()

    # any row not missing while the last was is a gap end        
    gap_ends = series[~missing & different_from_last].index

    # count the start as different from the last
    different_from_last[0] = True

    # any row missing while the last wasn't is a gap start
    gap_starts = series[missing & different_from_last].index        

    # check and remedy if series ends with missing data
    if len(gap_starts) > len(gap_ends):
         gap_ends = gap_ends.append(series.index[-1:] + series.index.freq)

    return pd.Series(index=gap_starts, data=gap_ends)

为了记录，Pandas == 0.13.1，Numpy == 1.8.1，Python 2.7

Answer 1

可以转换此问题以在列表中查找连续数字。找到系列为null的所有索引，如果（3,4,5,6）的运行都为null，则只需要提取开始和结束（3,6）

import numpy as np
import pandas as pd
from operator import itemgetter
from itertools import groupby


# create an example 
data = [2, 3, 4, 5, 12, 13, 14, 15, 16, 17]
s = pd.series( data, index=data)
s = s.reindex(xrange(18))
print find_gap(s)  


def find_gap(s): 
    """ just treat it as a list
    """ 
    nullindex = np.where( s.isnull())[0]
    ranges = []
    for k, g in groupby(enumerate(nullindex), lambda (i,x):i-x):
        group = map(itemgetter(1), g)
        ranges.append((group[0], group[-1]))
    startgap, endgap = zip(* ranges) 
    return pd.series( endgap, index= startgap )

参考：Identify groups of continuous numbers in a list

描述时间序列熊猫的差距

1 个答案: