为什么神经网络不学习？

Question

我正在训练带有简单数据集的神经网络。我尝试了参数，优化程序，学习率的不同组合……但是，即使经过20个时期，网络仍然无法学习任何内容。

我想知道下面的代码在哪里出问题了吗？

from tensorflow.keras.models import  Sequential, load_model
from tensorflow.keras.layers import Input, Dense, Flatten
from tensorflow import keras
from livelossplot import PlotLossesKeras
from keras.models import Model
from sklearn.datasets import make_classification
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import pandas as pd


seed = 42

X, y = make_classification(n_samples=100000, n_features=2, n_redundant=0, 
                           n_informative=2, random_state=seed)

print(f"Number of features: {X.shape[1]}")
print(f"Number of samples: {X.shape[0]}")


df = pd.DataFrame(np.concatenate((X,y.reshape(-1,1)), axis=1))
df.set_axis([*df.columns[:-1], 'Class'], axis=1, inplace=True)

df['Class'] = df['Class'].astype('int')
X = df.drop('Class', axis=1)
y = df['Class']

X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

print(f"Train set: {X_train.shape}")
print(f"Validation set: {X_val.shape}")


scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train.astype(np.float64))
X_val_scaled = scaler.transform(X_val.astype(np.float64))

inputs = Input(shape=X_train_scaled.shape[1:])
h0 = Dense(5, activation='relu')(inputs)
h1 = Dense(5, activation='relu')(h0)
preds = Dense(1, activation = 'sigmoid')(h1)

model = Model(inputs=inputs, outputs=preds)
opt = keras.optimizers.Adam(lr=0.0001)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
history = model.fit(X_train_scaled, y_train, batch_size=128, epochs=20, verbose=0,
                    validation_data=(X_val_scaled, y_val),
                    callbacks=[PlotLossesKeras()]) 

score_train = model.evaluate(X_train_scaled, y_train, verbose=0)
score_test = model.evaluate(X_val_scaled, y_val, verbose=0) 
print('Train score:', score_train[0])
print('Train accuracy:', score_train[1])
print('Test score:', score_test[0])
print('Test accuracy:', score_test[1])

代码产生以下类型的输出

Answer 1

您使用了错误的丢失功能，请更改此行

model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

例如，

model.compile(optimizer=opt, loss='mse', metrics=['accuracy'])

分类交叉熵需要一个热编码的y，这意味着每个课程必须有一个0或一个1。 MSE只是均方误差，因此它将起作用。但是您也可以尝试其他损失。

您的y：

[1,0,1]

一个热编码的y：

[[0,1], [1,0], [0,1]]