Conv1D不更新权重。 （全零）和测试输出总是相同，等于最后一层重量？

Question

我想用1D CNN来预测第二天的太阳能。时间序列数据分辨率为一小时，长度为一年。我正在使用第1天的数据训练模型以预测第2天.xtrain = day1，ytrain = day2，xtest = day3以预测第4天。

24小时数据输入 - ＆gt; CNN - ＆gt; 24小时输出

我已将数据训练了10天（样本）;然后提前4天预测，问题是无论输入是什么，CNN总是给出相同的输出值。然后，我检查了权重，只有输出图层权重非零。此外，输出值不会随着输入的不同而改变。

数据的链接：https://mega.nz/#!NpoTzIBJ!U5l8ToQgcJ6xif2tMjIrXuace3skhrtwLEdeoWe_FkM

重量矩阵的图像：

预测值图表：

代码：

import pandas
from pandas import Series
from pandas import DataFrame
import keras
from keras.callbacks import ModelCheckpoint
from sklearn.metrics import r2_score
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation, regularizers, initializers
from keras.layers import Conv1D,  MaxPooling1D, Flatten, AveragePooling1D
from keras.activations import *
from keras.losses import *
from keras.optimizers import *
from keras.utils import plot_model
from keras.models import Sequential
from keras.utils import plot_model
import numpy as np
import matplotlib.pyplot as plt
from sklearn import preprocessing
from numpy import zeros, newaxis
from keras.callbacks import EarlyStopping
import sklearn.metrics
from sklearn.metrics import mean_squared_error
from math import sqrt

data = pandas.read_csv("meas.csv", header=0)
dataset = data.values[:,1]

all_pred_data = []
ytest_all = []

model = Sequential()
model.add(Conv1D(3,kernel_size=3,activation='relu', input_shape=xtrain.shape[1:3],kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1),kernel_regularizer=regularizers.l2(0.1))) #input_shape=()
model.add(AveragePooling1D(pool_size=3))
model.add(Conv1D(3,kernel_size=3,activation='relu', input_shape=xtrain.shape[1:3],kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1),kernel_regularizer=regularizers.l2(0.1))) #input_shape=()
model.add(AveragePooling1D(pool_size=3))
model.add(Flatten())
model.add(Dense(42,activation='tanh',kernel_regularizer=regularizers.l2(0.1),kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1)))
model.add(Dense(24, activation='linear',kernel_regularizer=regularizers.l2(0.1),kernel_initializer=initializers.RandomUniform(minval=-1, maxval=1)))

model.compile(loss='mse',
          optimizer=keras.optimizers.Adam(),
          metrics=['mae','accuracy'])


for i in range(0,10-1,1): 

    xtrain = dataset[24*(i+1)-24:24*(i+1)]
    ytrain = dataset[24*(i+1):24*(i+2)]

    xtrain = xtrain.reshape(1,24,1)
    ytrain = ytrain.reshape(1,24)

    model.fit(xtrain,ytrain,epochs=500,verbose=2) 


# TEST
for i in range(20,25-1,1):
    xtest = dataset[24*(i+1):24*(i+2)] #(i+1):(i+6+1)
    ytest = dataset[24*(i+2):24*(i+2)+24]

    xtest = xtest.reshape(1,xtrain.shape[1], 1)

    pred_data = np.round(model.predict(xtest),3)

    pred_data_transpose = pred_data.transpose()

    all_pred_data.extend(pred_data_transpose)
    ytest_all.extend(np.round(ytest,3))

Answer 1

如果权重全部变为0，它们永远不会改变，因为渐变总是为0，以避免这种尝试将数据标准化，这样可以更容易地训练网络，如果规范化输入数据，还要尝试使用batch_normalization够了