Question

我正在尝试在tensorflow.js中实现自定义激活（Swish）功能。我该怎么办？

我没有找到有关自定义激活功能的任何信息，而是添加了custom layer。因此，我实现了一个自定义层，该层是在未分配任何激活的层之后手动添加的。它似乎可以工作，但是它在代码（尤其是模型定义）中非常麻烦。如何直接扩展激活类以获得可以在任何层定义内调用的激活函数？激活函数在TF.js代码的https://github.com/tensorflow/tfjs-layers/blob/master/src/activations.ts中定义。它应该像这样工作： const dense_layer = tf.layers.dense({units: 8, activation: my_Swish});

我将我的代码放在一起，在其中延伸了一层以起到激活功能的作用。这是班Swish。我还尝试通过Swich2类扩展tf.layers.activations，但调用它会产生错误。

errors.ts:48 Uncaught Error: activation: Improper config format: {"_callHook":null,"_addedWeightNames":[],"_stateful":false,"id":1,"activityRegularizer":null,"inputSpec":null,"supportsMasking":true,"_trainableWeights":[],"_nonTrainableWeights":[],"_losses":[],"_updates":[],"_built":false,"inboundNodes":[],"outboundNodes":[],"name":"activation_Activation1","trainable_":true,"initialWeights":null,"_refCount":null,"fastWeightInitDuringBuild":false,"activation":{}}.
'className' and 'config' must set.
    at new e (errors.ts:48)
    at Lp (generic_utils.ts:227)
    at rm (activations.ts:218)
    at im (activations.ts:239)
    at new e (core.ts:218)
    at Object.dense (exports_layers.ts:524)
    at tf.html:77

我已经在代码中添加了两个测试用例。只需取消注释第64和66行即可查看相应的错误。我的tf_test.html（可以在浏览器中本地运行）：

<html>
  <head>
    <!-- Load TensorFlow.js, Plotly for plots -->
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs/dist/tf.min.js"> </script>
      <script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
    <!-- Place your code in the script tag below. You can also use an external .js file -->    
  </head>

  <body>
  <div id="myDiv"><!-- Plotly chart will be drawn inside this DIV --></div>
  <script>
    // Notice there is no 'import' statement. 'tf' is available on the index-page
    // because of the script tag above.

    // custom layer
    // https://github.com/tensorflow/tfjs-examples/blob/master/custom-layer/custom_layer.js
    // https://gist.github.com/caisq/33ed021e0c7b9d0e728cb1dce399527d
    // https://github.com/tensorflow/tfjs-layers/blob/master/src/activations.ts
    class Swish extends tf.layers.Layer {
      //static className = 'Swish';
      constructor(config) {
        super(config);
        this.alpha = config.alpha;
      }

      call(input) {
        return tf.tidy(() => {
          const x = input[0]; //tf.getExactlyOneTensor(input);
          return tf.sigmoid(x.mul(this.alpha)).mul(x);
        });
      }

      computeOutputShape(inputShape){
        return inputShape;
      }

      static get className() {
      return 'Swish';
      }

      getConfig() {
        const config = {alpha: this.alpha};
        const baseConfig = super.getConfig();
        Object.assign(config, baseConfig);
        return config;
      }
    }

    tf.serialization.registerClass(Swish);

    class Swish2 extends tf.layers.activation {
      static get className() {return 'Swish2'};
      apply(x, alpha = 1) {
        return tf.sigmoid(x.mul(this.alpha)).mul(x);
      }
    }
    tf.serialization.registerClass(Swish2);

    const sw = new Swish({alpha: 1});
    const sw2 = new Swish2(1);
    // create Model
    const m_in = tf.input({shape: [1]});
    // this doesn't work!!! activations have to be defined manually as layers and then added, see below
    //const test = tf.layers.dense({units: 8, activation: sw}).apply(m_in);
    // sw2 can't be called this way either, yields 'Improper config format' error
    //const test2 = tf.layers.dense({units: 8, activation: sw2}).apply(m_in);
    const dense1 = tf.layers.dense({units: 8}).apply(m_in);
    const swa = sw.apply(dense1);
    const m_out = tf.layers.dense({units: 1}).apply(swa);
    const model = tf.model({inputs: m_in, outputs: m_out});

  </script>
  </body>
</html>

Answer 1

动机：配置格式不正确：{“ _callHook”：null，“ _ addedWeightNames”：[]，“ _ stateful”：false，“ id”：0，“ activityRegularizer”：null，“ inputSpec”：null，“ supportsMasking” ：true，“ _ trainableWeights”：[]，“ _ nonTrainableWeights”：[]，“ _ losses”：[]，“ _ updates”：[]，“ _ built”：false，“ inboundNodes”：[]，“ outboundNodes”：[] ，“名称”：“ activation_Activation1”，“ trainable _”：true，“ initialWeights”：null，“ _ refCount”：null，“ fastWeightInitDuringBuild”：false，“ activation”：{}，“ alpha”：1}。

该错误表明未设置className，也未配置配置，因为Activation类的getActivation返回一个空对象。查看代码here，我们可以看到，如果激活不是字符串，则返回没有类名和配置的空Layer。

设置自定义激活层的正确方法是将其应用为doc中指示的层。

在tensorflow.js中自定义激活（Swish）

1 个答案: