Question

有一些关于将L1正则化添加到一个隐藏的权重的答案。然而，我想要的不仅是权重的稀疏性，还有一个隐藏层的表示的稀疏性。我想要的是类似下面的代码。是否可行实现，或者我只需要在权重上添加L1正则化？

import tensorflow as tf
...
**HIDDEN** = tf.contrib.layers.dense(input_layer, n_nodes)
...
loss = meansq #or other loss calcuation
l1_regularizer = tf.contrib.layers.l1_regularizer(scale=0.005, scope=None)
regularization_penalty = tf.contrib.layers.apply_regularization(l1_regularizer, **HIDDEN**)

regularized_loss = loss + regularization_penalty

这个想法来自Goodfellow和Bengio所着的“深度学习”一书的稀疏表现。

Answer 1

如果您使用tf.contrib.layers，fully_connected函数会接受weights_regularizer参数，因此您的代码应该如此

l1 = tf.contrib.layers.l1_regularizer(scale=0.005, scope=None)
hidden = tf.contrib.layers.fully_connected(inputs, n_nodes, weights_regularizer=l1)

尽管如此，tf.contrib.layers大部分已移至核心API，因此您应该使用tf.layers.dense代替kernel_regularizer参数。

上面的代码将规范图层中的权重。如果要规范权重和图层输出，可以使用相同的tf.contrib.layers.l1_regularizer或使用不同的参数创建不同的l1 = tf.contrib.layers.l1_regularizer(scale=0.005, scope=None) hidden = tf.contrib.layers.fully_connected(inputs, n_nodes, weights_regularizer=l1) hidden_reg = l1(hidden)。这样的事情对你有用：

def get_oauth_token():
    resource_url = "https://xxxxxxxxxxx.co.ke/oauth/v1/generate?grant_type=client_credentials"
    key = "#################"
    secret = "################"
    r = requests.get(resource_url, auth=HTTPBasicAuth(key, secret))
    return r.json()['access_token']


def lipa_online(token):
    time = str(datetime.datetime.now()).split(".")[0].replace("-", "").replace(" ", "").replace(":", "")
    password = "{0}{1}{2}".format("174379", "bfb279f9aa9bdbcf158e97dd71a467cd2e0c893059b10f78e6b72ada1ed2c919", time)
    encoded = base64.b64encode(password)
    payload = {
      "BusinessShortCode": XXXXXX,
      "Password": encoded,
      "Timestamp": time,
      "TransactionType": "PayOnline",
      "Amount": 1,
      "PartyA": 25470000000,
      "PartyB": XXXXXX,
      "PhoneNumber": 25470000000,
      "CallBackURL": "https://97ff8c8c.ngrok.io/callback",
      "AccountReference": "13dbd6hg",
      "TransactionDesc": "Just a trial"
    }
    headers = {'Authorization': 'Bearer {0}'.format(token), 'Content-Type': "application/json"}
    r = requests.post("https://xxxxxxxx.co.ke/pay/stkpush/v1/processrequest", headers=headers, json=payload)
    print r.json()

if __name__ == "__main__":
    token = get_oauth_token()
    #The line below fires the payment action.
    request_id = pay_online(token)

如何将L1正则化添加到一个隐藏层？

1 个答案: