Question

我在LSTM模型中计算RMSE（均方根误差）时遇到了一些麻烦。该模型非常合适，并且我获得了良好的损失减少，但是当我尝试逆向转换我的yhat结果时，我收到以下错误：

non-broadcastable output operand with shape (399,1) doesn't match the broadcast shape (399,4)

这是我的代码：

预处理：

btc = pd.read_csv('live_bitcoin.csv')
twitter_sent = pd.read_csv('live_tweet.csv')
reddit_sent = pd.read_csv('live_reddit.csv')

btc.columns = ["price_usd","24h_volume_usd","market_cap_usd","available_supply","total_supply","percent_change_1h","percent_change_24h","percent_change_7d", "Sell", "Buy", "15m", "Stamp"]
twitter_sent.columns = ["Sentiment", "Stamp"]
reddit_sent.columns = ["Sentiment", "Stamp"]

merged = pd.merge(twitter_sent, btc,  on='Stamp', how='inner').merge(reddit_sent, on='Stamp', how='inner')
data = merged[["Sentiment_x", "Sentiment_y","24h_volume_usd", "market_cap_usd", "available_supply","price_usd"]].groupby(merged['Stamp']).mean()
datag = data[["24h_volume_usd", "market_cap_usd", "available_supply","price_usd"]]
tw_sentiment = data["Sentiment_x"]
rdt_sentiment = data["Sentiment_y"]

print "Dataset size: " + str(len(datag))
print "Timespan: " + str(len(datag)/60) + " hours"

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler(feature_range=(0, 1))
values = datag.values.reshape(-1, datag.shape[1])
tw_sentiment = tw_sentiment.values.reshape(-1, 1)
rdt_sentiment = rdt_sentiment.values.reshape(-1, 1)
tw_sentiment = tw_sentiment.astype('float32')
rdt_sentiment = rdt_sentiment.astype('float32')
values = values.astype('float32')
scaled = scaler.fit_transform(values)

训练：

train_size = int(len(scaled) * 0.7)
test_size = len(scaled) - train_size
train, test = scaled[0:train_size,:], scaled[train_size:len(scaled),:]
split = train_size

def create_dataset(dataset, look_back, tw_sentiment, rdt_sentiment, sent=False):
    dataX, dataY = [], []
    for i in range(len(dataset) - look_back):
        if i >= look_back:
            a = dataset[i-look_back:i+1, 0]
            a = a.tolist()
            if(sent==True):                
                current_tw_sentiment = tw_sentiment[i].tolist()[0]
                current_rdt_sentiment = rdt_sentiment[i].tolist()[0]
                a.append(current_tw_sentiment)
                a.append(current_rdt_sentiment)
            dataX.append(a)
            dataY.append(dataset[i + look_back, 0])
    print(len(dataY))
    return np.array(dataX), np.array(dataY)

look_back = 2
trainX, trainY = create_dataset(train, look_back, tw_sentiment[0:train_size], rdt_sentiment[0:train_size], sent=True)
testX, testY = create_dataset(test, look_back, tw_sentiment[train_size:len(scaled)], rdt_sentiment[train_size:len(scaled)], sent=True)

trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

# Creating new model
model = Sequential()
model.add(LSTM(100, input_shape=(trainX.shape[1], trainX.shape[2]), return_sequences=True))
model.add(LSTM(100))
model.add(Dense(1))
model.compile(loss='mae', optimizer='adam')
model.save('LSTM_14-03-2018.h5')

# Loading model
# model = load_model('models/LSTM_12-03-2018_GOOD.h5')

history = model.fit(trainX, trainY, epochs=300, batch_size=100, validation_data=(testX, testY), verbose=0, shuffle=False)
yhat = model.predict(testX)
yhat_inverse = scaler.inverse_transform(yhat.reshape(-1, 1))
testY_inverse = scaler.inverse_transform(testY.reshape(-1, 1))
rmse_sent = sqrt(mean_squared_error(testY_inverse, yhat_inverse))
print "Done"
print 'Test RMSE: %.3f' % rmse_sent

主要问题在于：

  yhat_inverse = scaler.inverse_transform(yhat.reshape(-1,1))
  testY_inverse = scaler.inverse_transform(testY.reshape(-1,1))

根据我的理解（仍然是ML中的初学者），我的yhat变量的形状为（399,1），因为我试图根据几个特征进行预测。我只想将我的数据恢复到之前的转换，因此RMSE错误以适当的比例返回。我基本上试图将价格重新调整到正常规模。在MinMaxScaler在预处理阶段完成数据后，我也永远不会重新转换数据。

任何可能出错的线索？

Answer 1

假设您的数据集名为df，并且您正在执行第（i）列的缩放，则必须看起来像这样：

df[list(df)[i]] = min_max_scaler.fit_transform(df[list(df) 
[i]].values.reshape(-1,1))

Answer 2

以下对我有用。

scaler2 = MinMaxScaler() scaler2.min_, scaler2.scale_ = scaler.min_[0], scaler.scale_[0]

yhat_inverse = scaler2.inverse_transform(yhat) testY_inverse = scaler2.inverse_transform(testY)

LSTM。使用inverse_transform

2 个答案: