如何在Python中将此峰发现进行矢量化处理？

Question

基本上，我正在编写一个峰值发现函数，该函数必须能够在基准测试中胜过scipy.argrelextrema。这是我正在使用的数据和代码的链接：

https://drive.google.com/open?id=1U-_xQRWPoyUXhQUhFgnM3ByGw-1VImKB

如果此链接失效，则可以在dukascopy银行的在线历史数据下载器中找到数据。

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

data = pd.read_csv('EUR_USD.csv')
data.columns = ['Date', 'open', 'high', 'low', 'close','volume']

data.Date = pd.to_datetime(data.Date, format='%d.%m.%Y %H:%M:%S.%f')

data = data.set_index(data.Date)

data = data[['open', 'high', 'low', 'close']]

data = data.drop_duplicates(keep=False)

price = data.close.values

def fft_detect(price, p=0.4):

    trans = np.fft.rfft(price)
    trans[round(p*len(trans)):] = 0
    inv = np.fft.irfft(trans)
    dy = np.gradient(inv)
    peaks_idx = np.where(np.diff(np.sign(dy)) == -2)[0] + 1
    valleys_idx = np.where(np.diff(np.sign(dy)) == 2)[0] + 1

    patt_idx = list(peaks_idx) + list(valleys_idx)
    patt_idx.sort()

    label = [x for x in np.diff(np.sign(dy)) if x != 0]

    # Look for Better Peaks

    l = 2

    new_inds = []

    for i in range(0,len(patt_idx[:-1])):

        search = np.arange(patt_idx[i]-(l+1),patt_idx[i]+(l+1))

        if label[i] == -2:
            idx = price[search].argmax()
        elif label[i] == 2:
            idx = price[search].argmin()

        new_max = search[idx]
        new_inds.append(new_max)

    plt.plot(price)
    plt.plot(inv)
    plt.scatter(patt_idx,price[patt_idx])
    plt.scatter(new_inds,price[new_inds],c='g')
    plt.show()

    return peaks_idx, price[peaks_idx]

基本上，它使用快速傅立叶变换（FFT）平滑数据，然后使用导数找到平滑数据的最小和最大索引，然后在未平滑数据上找到相应的峰值。有时，由于某些平滑效果，它发现的峰不清晰，因此我运行此for循环以搜索l指定的边界之间的每个索引的较高或较低的点。我需要向量化此for循环的帮助！我不知道该怎么做。没有for循环，我的代码比scipy.argrelextrema快50％，但是for循环使它变慢。因此，如果我能找到一种向量化的方法，它将是scipy.argrelextrema的非常快速且有效的替代方法。这两个图像分别表示没有for循环和带有interface MyType { id: string; value: number; } const myType: MyType = { id: '', value: 0 }; type ArrType<T> = Array<keyof T>; function isMyTypeArr<T>(arg: any[]): arg is ArrType<T> { return arg.length === Object.keys(myType).length; } function checkKeys<T>(arr: ArrType<T>): void { if (isMyTypeArr(arr)) { console.log(arr.length); // some other stuff } } checkKeys<MyType>(['id', 'x']); // TS error checkKeys<MyType>(['id']); // no console because of Type Guard checkKeys<MyType>(['id', 'value']); // SUCCESS: console logs '2' 循环的数据。

Answer 1

这可以做到。它不是完美的，但希望它能获得您想要的并向您展示如何进行矢量化。很高兴听到您想出的任何改进

label = np.array(label[:-1]) # not sure why this is 1 unit longer than search.shape[0]? 

# the idea is to make the index matrix you're for looping over row by row all in one go. 
# This part is sloppy and you can improve this generation. 

search = np.vstack((np.arange(patt_idx[i]-(l+1),patt_idx[i]+(l+1)) for i in range(0,len(patt_idx[:-1])))) # you can refine this. 

# then you can make the price matrix

price = price[search]

# and you can swap the sign of elements so you only need to do argmin instead of both argmin and argmax 

price[label==-2] = - price[label==-2]

# now find the indices of the minimum price on each row 

idx = np.argmin(price,axis=1)

# and then extract the refined indices from the search matrix 

new_inds = search[np.arange(idx.shape[0]),idx] # this too can be cleaner. 
# not sure what's going on here so that search[:,idx] doesn't work for me
# probably just a misunderstanding 

我发现这可以重现您的结果，但是我没有计时。我怀疑搜索的生成速度很慢，但可能仍比for循环快。

编辑：

这是产生search的更好方法：

patt_idx = np.array(patt_idx)
starts = patt_idx[:-1]-(l+1)
stops = patt_idx[:-1]+(l+1)
ds = stops-starts
s0 = stops.shape[0]
s1 = ds[0]
search = np.reshape(np.repeat(stops - ds.cumsum(), ds) + np.arange(ds.sum()),(s0,s1))

Answer 2

这是一种替代方法...它使用列表理解，通常比for循环快

l = 2

# Define the bounds beforehand, its marginally faster than doing it in the loop
upper = np.array(patt_idx) + l + 1
lower = np.array(patt_idx) - l - 1

# List comprehension...
new_inds = [price[low:hi].argmax() + low if lab == -2 else 
            price[low:hi].argmin() + low 
            for low, hi, lab in zip(lower, upper, label)]

# Find maximum within each interval
new_max = price[new_inds]
new_global_max = np.max(new_max)