Question

我有一个带有speed列的Pandas数据框，其中偶尔会有噪音（数据来自Garmin并代表在运行期间捕获的数据）。

我试图找到一种平均相邻点的方法，但是当我点击这样的东西时

9.112273445
164.5779550738
84.4553498412
4.231089359
4.3740439706

我陷入无限循环。

我的算法很天真：

# Get list of indices in which value is great than 6:
idx = z[(z['speed']>=6)].index
while list(idx) != []:
    for i in idx:
        # check if out of bounds
        if i + 1 >= len(z):
            z.iloc[i, z.columns.get_indexer(['speed'])] = (z['speed'].ix[i-2] + z['speed'].ix[i-1])/2
        elif i - 1 < 0:
            z.iloc[i, z.columns.get_indexer(['speed'])] = (z['speed'].ix[i+1] + z['speed'].ix[i+2])/2
        else:
            z.iloc[i, z.columns.get_indexer(['speed'])] = (z['speed'].ix[i-1] + z['speed'].ix[i+1])/2
    idx = z[(z['speed']>=6)].index

当然，问题是当我有两个非常大的相邻值时，会陷入无限循环。

我正在应用此过滤器（使用汉宁窗口）消除随机噪音：SciPy Cookbook SignalSmooth，但它没有处理数据中的这些大峰值。

如果没有丢弃它们，或者将它们设置为常数值，还有其他简单的处理方法吗？

修改

我测试的价值是：

0           NaN
1      3.508394
2      5.097879
3      7.743824
4      9.138245
5     13.315918
6     12.836310
7     12.001393
8     15.815223
9      0.000000
10    16.622944
11     9.061864
12     2.089729
13     2.710874
Name: speed, dtype: float64

Answer 1

如果你想“桥接”大于6的值，你可以这样做：

import numpy as np

# locate outliers and adjacent values
outliers = np.r_[False, (~np.isfinite(data)) | (data > 6), False]
if np.any(outliers):
    boundaries = np.where(outliers[:-1] != outliers[1:])[0]
    lb = boundaries[::2]
    rb = boundaries[1::2]
    # special case if leftmost and/or rightmost values are outliers 
    lv = data[lb-1]
    if lb[0] == 0:
        lv[0] = data[rb[0]]
    rv = data[rb % len(data)]
    if rb[-1] == len(data):
        rv[-1] = data[lb[-1]-1]
    # create fill values; use a bit of trickery to keep it vectorised
    lengths = rb-lb
    fv = np.repeat((rv-lv)/(lengths+1), lengths)
    sw = np.cumsum(lengths[:-1])
    fv[sw] += fv[sw-1] - rv[:-1] + lv[1:]
    fv[0] += lv[0]
    fv = np.cumsum(fv)
    # place them
    out = data.copy()
    out[outliers[1:-1]] = fv
else:
    out = data.copy()

过滤掉gpx数据中的噪音

1 个答案: