Keras重量缩放 - 计算图

Question

型号代码：

Project\File.cs

问题：

Keras版本：1.0.1

• 7-8行只是在开始时或每次发生一次？我的意思是 它添加到计算图中，是由权重缩放的 每次都是wt_scale吗？

打印图表的代码：

vfnet = Sequential()
for (i, layeroutsize) in enumerate(hid_sizes):
    inshp = dict(input_shape=(ob_space.shape[0]+1,)) if i==0 else {} # add one extra feature for timestep
    vfnet.add(Dense(layeroutsize, activation=cfg["activation"], init="glorot_uniform",**inshp))
vfnet.add(Dense(1))

for layer in vfnet.layers:
    layer.W.set_value(layer.W.get_value(borrow=True)*wt_scale)

图形（混淆!!）：

import theano
for i in vfnet.layers:
    theano.printing.debugprint(i.input)
    theano.printing.debugprint(i.output)
ipdb.set_trace()

可视化：

Using Theano backend.
[2017-07-22 17:07:27,139] Making new env: CartPole-v0
dense_input_2 [id A]
Elemwise{tanh,no_inplace} [id A] ''
 |Elemwise{add,no_inplace} [id B] ''
   |dot [id C] ''
   | |dense_input_2 [id D]
   | |dense_4_W [id E]
   |InplaceDimShuffle{x,0} [id F] ''
     |dense_4_b [id G]
Elemwise{tanh,no_inplace} [id A] ''
 |Elemwise{add,no_inplace} [id B] ''
   |dot [id C] ''
   | |dense_input_2 [id D]
   | |dense_4_W [id E]
   |InplaceDimShuffle{x,0} [id F] ''
     |dense_4_b [id G]
Elemwise{tanh,no_inplace} [id A] ''
 |Elemwise{add,no_inplace} [id B] ''
   |dot [id C] ''
   | |Elemwise{tanh,no_inplace} [id D] ''
   | | |Elemwise{add,no_inplace} [id E] ''
   | |   |dot [id F] ''
   | |   | |dense_input_2 [id G]
   | |   | |dense_4_W [id H]
   | |   |InplaceDimShuffle{x,0} [id I] ''
   | |     |dense_4_b [id J]
   | |dense_5_W [id K]
   |InplaceDimShuffle{x,0} [id L] ''
     |dense_5_b [id M]
Elemwise{tanh,no_inplace} [id A] ''
 |Elemwise{add,no_inplace} [id B] ''
   |dot [id C] ''
   | |Elemwise{tanh,no_inplace} [id D] ''
   | | |Elemwise{add,no_inplace} [id E] ''
   | |   |dot [id F] ''
   | |   | |dense_input_2 [id G]
   | |   | |dense_4_W [id H]
   | |   |InplaceDimShuffle{x,0} [id I] ''
   | |     |dense_4_b [id J]
   | |dense_5_W [id K]
   |InplaceDimShuffle{x,0} [id L] ''
     |dense_5_b [id M]
Elemwise{add,no_inplace} [id A] ''
 |dot [id B] ''
 | |Elemwise{tanh,no_inplace} [id C] ''
 | | |Elemwise{add,no_inplace} [id D] ''
 | |   |dot [id E] ''
 | |   | |Elemwise{tanh,no_inplace} [id F] ''
 | |   | | |Elemwise{add,no_inplace} [id G] ''
 | |   | |   |dot [id H] ''
 | |   | |   | |dense_input_2 [id I]
 | |   | |   | |dense_4_W [id J]
 | |   | |   |InplaceDimShuffle{x,0} [id K] ''
 | |   | |     |dense_4_b [id L]
 | |   | |dense_5_W [id M]
 | |   |InplaceDimShuffle{x,0} [id N] ''
 | |     |dense_5_b [id O]
 | |dense_6_W [id P]
 |InplaceDimShuffle{x,0} [id Q] ''
   |dense_6_b [id R]

Answer 1

  

第7-8行是在开始时还是每次都发生一次？我的意思是它是否被添加到计算图中，每次都是通过wt_scale缩放的权重？

他们肯定只发生过一次。您正在访问权重一次并将它们乘以wt_scale。之后，所有培训都将正常进行。