Question

我正在尝试训练卷积神经网络，并且遇到了张量流的一些问题。我的进程被杀死并抛出以下错误：

2018-03-14 15:56:57.850610: W tensorflow/core/framework/op_kernel.cc:1142] Internal: Dst tensor is not initialized.
    /var/spool/slurmd/job06322/slurm_script: line 23: 416703 Killed

在查找问题后，我发现它与内存有关。集群中的节点有一个带有93GB RAM的gpu分区。所以，有一个内存问题似乎有点奇怪。我认为我分割数据集的方式有误，但我不太确定。我在下面列出了我正在做的事情的代码片段。如果我应该提供有关我正在使用的数据的任何其他信息，这将有助于解决问题，我可以提供它。

comments = pd.read_csv('set1.tsv', sep = '\t', index_col = 0)
annotations = pd.read_csv('set2.tsv',  sep = '\t')
labels = annotations.groupby('rev_id')['score'].mean()
labels = pd.DataFrame(data=labels.values, columns=['score'])
scaler = MinMaxScaler(feature_range=(0, 1))
labels = 1-(scaler.fit_transform(labels))
comments['score'] = labels
comments['comment'] = comments['comment'].apply(lambda x: x.replace("NEWLINE_TOKEN", " "))
comments['comment'] = comments['comment'].apply(lambda x: x.replace("TAB_TOKEN", " "))
comments = comments.reset_index()
vocab_size = 20000
max_length = 300  # max length of the comments in the model
tok = Tokenizer(num_words=vocab_size)  # make the Keras tokenizer
tok.fit_on_texts(comments['comment'])  # fit it to comments
train = comments.query("split=='train'")
test = comments.query("split=='test'")
x_train = pad_sequences(tok.texts_to_matrix(train[['comment']], mode='count'), maxlen=max_length, padding='post')
y_train = np.array([[float(d)] for d in train[['score']].pop('score')]).T
x_test = pad_sequences(tok.texts_to_matrix(test[['comment']], mode='count'), maxlen=max_length, padding='post')
y_test = np.array([[float(d)] for d in test[['score']].pop('score')]).T
embedding_vecor_length = 128
model = Sequential()
model.add(Embedding(vocab_size, embedding_vecor_length, input_length=max_length))
model.add(Convolution1D(64, 3, padding='same'))
model.add(Convolution1D(32, 3, padding='same'))
model.add(Convolution1D(16, 3, padding='same'))
model.add(Flatten())
model.add(Dropout(0.2))
model.add(Dense(95692,activation='sigmoid'))
model.add(Dropout(0.2))
model.add(Dense(95692,activation='sigmoid'))
tensorBoardCallback = TensorBoard(log_dir='./logs', write_graph=True)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, epochs=3, callbacks=[tensorBoardCallback])

Tensorflow - Dst张量未初始化

0 个答案: