Question

我正在使用当前规范化输入张量的LuaTorch normalization图层。我将其添加为网络的一部分，如self:add( nn.Normalize(2) )。现在我想仅规范化输入张量的一部分。我不确定如何在以下行中仅指定张量的一部分。

self:add( nn.View(-1, op_neurons) )
self:add( nn.Normalize(2) )  <--- how to normalize only a part of the input tensor
self:add( nn.View(-1,no_of_objects,op_neurons) )

Answer 1

我认为干净的方法是从nn.Normalize派生自己的类。只需创建一个像PartialNormalize.lua这样的文件，然后继续这样做（这很容易，但开发时间有点费时，所以我只是主要给你伪代码）：

local PartialNormalize, parent = torch.class('nn.PartialNormalize', 'nn.Normalize')
--now basically you need to override the funcions __init, updateOutput and updateGradInput from the parent class (I dont think there is a need to override other functions, but you shoud make check.)
-- you can find the code for nn.Normalize in  <your_install_path>/install/share/lua/5.1/nn/Normalize.lua
-- the interval [first_index, last_index] determines which parts  from your input vector you want to be normalized.

function PartialNormalize:__init(p,eps,first_index,last_index)
  parent.__init(self) 
  self.first_index=first_index
  self.last_index=last_index
end


function PartialNormalize:updateOutput(input)
  --In the parent class, this just returns the normalized part 
  -- just modify this function so that it returns the normalized part from self.first_index to self.last_index, and that it just passes the other elements through
end 

function PartialNormalize:updateGradInput(input, gradOutput)
    -- make appropriate modifications to the gradient function: gradient for elements from self.first_index to self.last_index is computed just as in the parent class,
    -- while the gradient for other elements is just 1 everywhere
end

   -- I don't think other functions from the parent class need overriding, but make sure just in case

希望这有帮助。

Answer 2

有用于独立处理输入部件的容器。使用concat和narrow，您可以构建部分规范化。

require"torch"
nn=require"nn"
local NX,NY = 2,6 --sizes of inputs
local Y1, Y2 = 1, 4 --normalize only data between these constraints
local DIMENSION_INDEX=2--dimension on which you want to split your input (NY here)
local input=torch.randn(NX,NY)--example input

--network construction
local normalize_part = nn.Sequential()
normalize_part:add(nn.Narrow(DIMENSION_INDEX,Y1,Y2))
normalize_part:add(nn.Normalize(2))


local dont_change_part=nn.Sequential()
dont_change_part:add(nn.Narrow(DIMENSION_INDEX,Y2+1,NY-Y2))


local partial_normalization=nn.Concat(DIMENSION_INDEX)
partial_normalization:add(normalize_part)
partial_normalization:add(dont_change_part)

--partial_normalization is ready for use:
print(input)
print( partial_normalization:forward(input))

--can be used as a block in a greater network
local main_net=nn.sequential()
main_net:add(partial_normalization)

此外，我还要注意nn.normalize不等同于(X - mean(x)) / std(x)，也称为规范化。

Lua Torch标准化层仅标准化张量的一部分

2 个答案: