父作用域中的模型可防止子进程范围内的模型

时间:2017-06-22 06:35:23

标签: python tensorflow keras python-multiprocessing

所以这就是我面临的问题。我试图在多处理Process中训练模型,但是当父范围中已存在模型时,该过程将在Embedding层的初始化时冻结。

from multiprocessing import Process, Pipe
import numpy as np
from keras.models import Model
from keras.layers import Input, Dense, Embedding
from keras.optimizers import Adam
import tensorflow as tf


def make_model(vecs, weights=None):
    inp = Input((5,))
    embd = Embedding(len(vecs), 50, weights=[vecs], trainable=False)(inp)
    out = Dense(5, activation='softmax')(embd)
    model = Model(inp, out)
    model.compile(Adam(0.001), 'categorical_crossentropy', metrics=['accuracy'])
    return model


def f(vecs, conn):
    model = make_model(vecs)
    conn.send('done')
    conn.close()


if __name__ == '__main__':
    vecs = np.random.random((100000, 50))
    model1 = make_model(vecs)

    parent_conn, child_conn = Pipe()
    p = Process(target=f, args=(vecs, child_conn), daemon=True)
    p.start()

    print('starting model two')
    print(parent_conn.recv())
    print('completed')

当此脚本按其当前编写的方式运行时,它将永远不会打印出已完成的脚本'信息。但是,如果我注释掉model1 = make_model(vecs)行,那么它就可以了。

0 个答案:

没有答案