如何使用自动编码器初始化MLP的权重＃2nd part - Deep autoencoder＃3rd part - Stacked autoencoder

Question

我已经构建了一个自动编码器（1个编码器8：5,1个解码器5：8），它采用Pima-Indian-Diabetes数据集（https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv）并减小其维度（从8到5）。我现在想使用这些缩减的功能使用mlp对数据进行分类。现在，在这里，我对架构的基本理解存在一些问题。如何使用自动编码器的权重并将其输入mlp？我检查了这些主题 - https://github.com/keras-team/keras/issues/91和https://www.codementor.io/nitinsurya/how-to-re-initialize-keras-model-weights-et41zre2g。这里的问题是我应该考虑哪个权重矩阵？编码器部分或解码器部分？当我为mlp添加图层时，如何使用这些保存的权重初始化权重，而不是获得确切的语法。另外，我的mlp应该从5个神经元开始，因为我的尺寸减小了5？对于这个二进制分类问题，mlp的可能维度是多少？如果有人可以详细说明吗？

深度自动编码器代码如下：

# from keras.models import Sequential
from keras.layers import Input, Dense
from keras.models import Model
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
import numpy

# Data pre-processing...

# load pima indians dataset
dataset = numpy.loadtxt("C:/Users/dibsa/Python Codes/pima.csv", delimiter=",")
# split into input (X) and output (Y) variables
X = dataset[:, 0:8]
Y = dataset[:, 8]

# Split data into training and testing datasets
x_train, x_test, y_train, y_test = train_test_split(
                                X, Y, test_size=0.2, random_state=42)

# scale the data within [0-1] range
scalar = MinMaxScaler()
x_train = scalar.fit_transform(x_train)
x_test = scalar.fit_transform(x_test)

# Autoencoder code begins here...
encoding_dim1 = 5    # size of encoded representations
encoding_dim2 = 3    # size of encoded representations in the bottleneck layer

# this is our input placeholder
input_data = Input(shape=(8,))
# "encoded" is the first encoded representation of the input
encoded = Dense(encoding_dim1, activation='relu', name='encoder1')(input_data)
# "enc" is the second encoded representation of the input
enc = Dense(encoding_dim2, activation='relu', name='encoder2')(encoded)
# "dec" is the lossy reconstruction of the input
dec = Dense(encoding_dim1, activation='sigmoid', name='decoder1')(enc)
# "decoded" is the final lossy reconstruction of the input
decoded = Dense(8, activation='sigmoid', name='decoder2')(dec)
# this model maps an input to its reconstruction
autoencoder = Model(inputs=input_data, outputs=decoded)

autoencoder.compile(optimizer='sgd', loss='mse')

# training
autoencoder.fit(x_train, x_train,
            epochs=300,
            batch_size=10,
            shuffle=True,
            validation_data=(x_test, x_test))  # need more tuning

# test the autoencoder by encoding and decoding the test dataset
reconstructions = autoencoder.predict(x_test)
print('Original test data')
print(x_test)
print('Reconstructed test data')
print(reconstructions)

#The stacked autoencoder code is as follows:

# from keras.models import Sequential
from keras.layers import Input, Dense
from keras.models import Model
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
import numpy

# Data pre-processing...

# load pima indians dataset
dataset = numpy.loadtxt("C:/Users/dibsa/Python Codes/pima.csv", delimiter=",")
# split into input (X) and output (Y) variables
X = dataset[:, 0:8]
Y = dataset[:, 8]

# Split data into training and testing datasets
x_train, x_test, y_train, y_test = train_test_split(
                                X, Y, test_size=0.2, random_state=42)

# scale the data within [0-1] range
scalar = MinMaxScaler()
x_train = scalar.fit_transform(x_train)
x_test = scalar.fit_transform(x_test)

# Autoencoder code goes here...
encoding_dim1 = 5    # size of encoded representations
encoding_dim2 = 3    # size of encoded representations in the bottleneck layer

# this is our input placeholder
input_data1 = Input(shape=(8,))
# the first encoded representation of the input
encoded1 = Dense(encoding_dim1, activation='relu',
             name='encoder1')(input_data1)
# the first lossy reconstruction of the input
decoded1 = Dense(8, activation='sigmoid', name='decoder1')(encoded1)
# this model maps an input to its first layer of reconstructions
autoencoder1 = Model(inputs=input_data1, outputs=decoded1)
# this is the first encoder model
enc1 = Model(inputs=input_data1, outputs=encoded1)

autoencoder1.compile(optimizer='sgd', loss='mse')

# training
autoencoder1.fit(x_train, x_train, epochs=300,
             batch_size=10, shuffle=True,
             validation_data=(x_test, x_test))
FirstAEoutput = autoencoder1.predict(x_train)

input_data2 = Input(shape=(encoding_dim1,))
# the second encoded representations of the input
encoded2 = Dense(encoding_dim2, activation='relu',
             name='encoder2')(input_data2)
# the final lossy reconstruction of the input
decoded2 = Dense(encoding_dim1, activation='sigmoid',
             name='decoder2')(encoded2)

# this model maps an input to its second layer of reconstructions
autoencoder2 = Model(inputs=input_data2, outputs=decoded2)

# this is the second encoder
enc2 = Model(inputs=input_data2, outputs=encoded2)

autoencoder2.compile(optimizer='sgd', loss='mse')

# training
autoencoder2.fit(FirstAEoutput, FirstAEoutput, epochs=300,
             batch_size=10, shuffle=True)

# this is the overall autoencoder mapping an input to its final reconstructions
autoencoder = Model(inputs=input_data1, outputs=encoded2)
# test the autoencoder by encoding and decoding the test dataset

reconstructions = autoencoder.predict(x_test)
print('Original test data')
print(x_test)
print('Reconstructed test data')
print(reconstructions)

Answer 1

这么多问题。你都尝试了些什么？代码片段？

如果您的解码器正在尝试重建输入，那么将分类器附加到其输出对我来说真的没有意义。我的意思是，为什么不在第一次将它附加到输入？因此，如果您开始使用自动编码器，我会非常清楚您应该将分类器附加到编码器管道的输出。

我不太清楚你的意思＆＃34;使用自动编码器的权重并将它们输入mlp＆＃34;。您不会使用其他图层的权重来提供图层，但需要使用输出信号。这对Keras来说非常容易。我们假设您定义了自动编码器并对其进行了如下培训：

(1, )

然后，您可以在编码器处附加分类层，并使用以下代码创建分类器模型：

from keras Input, Model
from keras import backend as K
from keras.layers import Dense

x = Input(shape=[8])
y = Dense(5, activation='sigmoid' name='encoder')(x)
y = Dense(8, name='decoder')(y)

ae = Model(inputs=x, outputs=y)
ae.compile(loss='mse', ...)
ae.fit(x_train, x_train, ...)

K.models.save_model(ae, './autoencoder.h5')

真的，真的。 # load the model from the disk if you # are in a different execution. ae = K.models.load_model('./autoencoder.h5') y = ae.get_layer('encoder').output y = Dense(1, activation='sigmoid', name='predictions')(y) classifier = Model(inputs=ae.inputs, outputs=y) classifier.compile(loss='binary_crossentropy', ...) classifier.fit(x_train, y_train, ...) 模型现在将classifier模型的第一个嵌入层编码器作为其第一层，然后是ae决策层预测< / em>的

如果您真正想要做的是使用自动编码器学习的权重来初始化分类器的权重（我不推荐这种方法）：

您可以使用sigmoid获取权重矩阵，修剪它（因为编码器有5个单位且分类器只有1个），最后设置分类器权重。以下几行：

layer#get_weights

Answer 2

Idavid，这是供您参考的 - MLP using Autoencoder reduced features。我需要了解哪个数字是正确的？对不起，由于没有通过评论上传图片的选项，我不得不上传图片作为答案。我想你是说图B是正确的。这是相同的代码片段。如果我走对了，请告诉我。

# This is a mlp classification code with features reduced by an Autoencoder

# from keras.models import Sequential
from keras.layers import Input, Dense
from keras.models import Model
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
import numpy

# Data pre-processing...

# load pima indians dataset
dataset = numpy.loadtxt("C:/Users/dibsa/Python Codes/pima.csv", delimiter=",")
# split into input (X) and output (Y) variables
X = dataset[:, 0:8]
Y = dataset[:, 8]

# Split data into training and testing datasets
x_train, x_test, y_train, y_test = train_test_split(
                                X, Y, test_size=0.2, random_state=42)

# scale the data within [0-1] range
scalar = MinMaxScaler()
x_train = scalar.fit_transform(x_train)
x_test = scalar.fit_transform(x_test)

# Autoencoder code goes here...
encoding_dim = 5    # size of our encoded representations

# this is our input placeholder
input_data = Input(shape=(8,))
# "encoded" is the encoded representation of the input
encoded = Dense(encoding_dim, activation='relu', name='encoder')(input_data)
# "decoded" is the lossy reconstruction of the input
decoded = Dense(8, activation='sigmoid', name='decoder')(encoded)
# this model maps an input to its reconstruction
autoencoder = Model(inputs=input_data, outputs=decoded)

autoencoder.compile(optimizer='sgd', loss='mse')

# training
autoencoder.fit(x_train, x_train,
            epochs=300,
            batch_size=10,
            shuffle=True,
            validation_data=(x_test, x_test))  # need more tuning

# test the autoencoder by encoding and decoding the test dataset
reconstructions = autoencoder.predict(x_test)
print('Original test data')
print(x_test)
print('Reconstructed test data')
print(reconstructions)

# MLP code goes here...
# create model

x = autoencoder.get_layer('encoder').output
# h = Dense(3, activation='relu', name='hidden')(x)
y = Dense(1, activation='sigmoid', name='predictions')(x)
classifier = Model(inputs=autoencoder.inputs, outputs=y)

# Compile model
classifier.compile(loss='binary_crossentropy', optimizer='adam',
               metrics=['accuracy'])

# Fit the model
classifier.fit(x_train, y_train, epochs=250, batch_size=10)

print('Now making predictions')
predictions = classifier.predict(x_test)
# round predictions
rounded_predicted_classes = [round(x[0]) for x in predictions]
temp = sum(y_test == rounded_predicted_classes)
acc = temp/len(y_test)
print(acc)