Question

CNTK中是否有效处理共享层？（即激活计算不重复）

示例假设我有以下表达式：

def create_func(shared_layers, out1_layers, out2_layers):

    # ... there would be a with block specifying activations...omitted for brevity
    shared_hl_func = For(shared_layers, lambda n: Dense(n), name="Shared Hidden Layers")
    out1_hl_func = For(out1_layers, lambda n: Dense(n), name="Out1 Only Hidden Layers")
    out2_hl_func = For(out2_layers, lambda n: Dense(n), name="Sigma Only Hidden Layers")

    output1_func = Sequential([shared_hl_func, out1_hl_func,
                                  Dense(1, activation=None, init=init, name="Out1_Regression_Layer")], name="Out1")
    output2_func = Sequential([shared_hl_func, out2_hl_func,
                                Dense(1, activation=None, init=init, name="Out2_Regression_Layer")], name="Out2")
    return output1_func, output2_func

output1, output2 = create_func([50,25], [25, 10], [25, 10])
my_input = cntk.input_variable((70,))
dual_model = cntk.combine(output1(my_input), output2(my_input))

评估dual_model时，计算是否会有效完成？（即，前两个较宽的密集层只会被计算一次然后共享吗？如果不是这样的话，那么将通过显式函数组合帮助提高效率吗？

Answer 1

在上面的代码中，shared_hl_func在output1_func和output2_func中独立评估，但参数是共享的。要检查计算图，请使用plot对其进行可视化。

要实现计算共享，需要将shared_hl_func输出变量传递给output1_func和output2：

import cntk
from cntk.layers import *
def create_func(shared_layers, out1_layers, out2_layers):
    shared_hl_func = For(shared_layers, lambda n: Dense(n), name="Shared Hidden Layers")
    out1_hl_func = For(out1_layers, lambda n: Dense(n), name="Out1 Only Hidden Layers")
    out2_hl_func = For(out2_layers, lambda n: Dense(n), name="Sigma Only Hidden Layers")
    out1_regr_func = Dense(1, activation=None, name="Out1_Regression_Layer")
    out2_regr_func = Dense(1, activation=None, name="Out2_Regression_Layer")
    @cntk.Function
    def _func(x):
        # ... there would be a with block specifying activations...omitted for brevity
        shared_hl = shared_hl_func(x)
        output1 = Sequential([out1_hl_func, out1_regr_func], name="Out1")(shared_hl)
        output2 = Sequential([out2_hl_func, out2_regr_func], name="Out2")(shared_hl)
        return cntk.combine(output1, output2)
    return _func

output = create_func([50,25], [25, 10], [25, 10])
my_input = cntk.input_variable((70,))
dual_model = output(my_input)
# use plot to visualize the model
cntk.logging.graph.plot(dual_model, 'dual.pdf')

共享层是否有效处理？

1 个答案: