如何用keras

Question

作为一项实验，我正在建立一个keras模型来逼近矩阵的行列式。然而，当我运行它时，损失在每个时代都会下降，并且验证损失会增加！例如：

8s - loss: 7573.9168 - val_loss: 21831.5428
Epoch 21/50
8s - loss: 7345.0197 - val_loss: 23594.8540
Epoch 22/50
13s - loss: 7087.7454 - val_loss: 24718.3967
Epoch 23/50
7s - loss: 6851.8714 - val_loss: 25624.8609
Epoch 24/50
6s - loss: 6637.8168 - val_loss: 26616.7835
Epoch 25/50
7s - loss: 6446.8898 - val_loss: 28856.9654
Epoch 26/50
7s - loss: 6255.7414 - val_loss: 30122.7924
Epoch 27/50
7s - loss: 6054.5280 - val_loss: 32458.5306
Epoch 28/50

以下是完整的代码：

import numpy as np
import sys
from scipy.stats import pearsonr
from scipy.linalg import det
from sklearn.model_selection import train_test_split
from tqdm import tqdm
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
import math
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from keras import backend as K

def baseline_model():
# create model
        model = Sequential()
        model.add(Dense(200, input_dim=n**2, kernel_initializer='normal', activation='relu'))
        model.add(Dense(1, input_dim=n**2))
        #        model.add(Dense(1, kernel_initializer='normal'))
        # Compile model
        model.compile(loss='mean_squared_error', optimizer='adam')
        return model


n = 15

print("Making the input data using seed 7", file=sys.stderr)
np.random.seed(7)
U = np.random.choice([0, 1], size=(n**2,n))
#U is a random orthogonal matrix
X =[]
Y =[]
# print(U)
for i in tqdm(range(100000)):
        I = np.random.choice(n**2, size = n)
        # Pick out the random rows and sort the rows of the matrix lexicographically.
        A = U[I][np.lexsort(np.rot90(U[I]))] 
        X.append(A.ravel())
        Y.append(det(A))

X = np.array(X)
Y = np.array(Y)

print("Data created")

estimators = []
estimators.append(('standardize', StandardScaler()))
estimators.append(('mlp', KerasRegressor(build_fn=baseline_model, epochs=50, batch_size=32, verbose=2)))
pipeline = Pipeline(estimators)
X_train, X_test, y_train, y_test = train_test_split(X, Y,
                                                    train_size=0.75, test_size=0.25)
pipeline.fit(X_train, y_train, mlp__validation_split=0.3)

  

如何阻止它过度拟合呢？

更新1

我尝试添加更多图层和L_2正则化。但是，它几乎没有差别。

def baseline_model():
# create model
        model = Sequential()
        model.add(Dense(n**2, input_dim=n**2, kernel_initializer='glorot_normal', activation='relu'))
        model.add(Dense(int((n**2)/2.0), kernel_initializer='glorot_normal', activation='relu', kernel_regularizer=regularizers.l2(0.01)))
        model.add(Dense(int((n**2)/2.0), kernel_initializer='glorot_normal', activation='relu', kernel_regularizer=regularizers.l2(0.01)))
        model.add(Dense(int((n**2)/2.0), kernel_initializer='glorot_normal', activation='relu', kernel_regularizer=regularizers.l2(0.01)))
        model.add(Dense(1, kernel_initializer='glorot_normal'))
        # Compile model
        model.compile(loss='mean_squared_error', optimizer='adam')
        return model

我将时期数增加到100，结束时：

19s - loss: 788.9504 - val_loss: 18423.2807
Epoch 97/100
24s - loss: 760.2046 - val_loss: 18305.9273
Epoch 98/100
20s - loss: 806.0941 - val_loss: 18174.8706
Epoch 99/100
24s - loss: 780.0487 - val_loss: 18356.7482
Epoch 100/100
27s - loss: 749.2595 - val_loss: 18331.5859

  

是否可以使用keras来近似矩阵的行列式？

Answer 1

我测试了你的代码并获得了相同的结果。但是让我们对矩阵行列式（DET）进行基本的理解。 DET由n组成！产品，所以你不能用几层神经网络中的n * n权重来近似它。这需要的数量不会扩展到n = 15，因为15！在DET中有1307674368000个乘法项。