Question

我正在尝试在Kernel Ridge回归中测试一个新的内核方法，并希望通过实现Fastfood转换（https://arxiv.org/abs/1408.3060）来实现这一点。我可以编写一个函数来计算这个转换，但它与sklearn中的内核岭回归函数不能很好地协作。因此，我转到了sklearn内核岭回归（https://insight.io/github.com/scikit-learn/scikit-learn/blob/master/sklearn/kernel_ridge.py）和approximate_kernel.py（https://insight.io/github.com/scikit-learn/scikit-learn/blob/master/sklearn/kernel_approximation.py）的源代码，以尝试将此新内核定义为approximate_kernel.py中的类定义。问题是我不知道如何将我的构造转换为可在approximate_kernel KernelRidge程序中工作的东西。是否有人能够建议如何做到最好？

我对快餐改造的构建是：

def fastfood_product(d):
'''
Constructs the fastfood matrix composition V = const*S*H*G*Pi*B where
S is a scaling matrix
H is Hadamard transform
G is a diagonal random Gaussian
Pi is a permutation matrix
B is a diagonal Rademacher matrix.
Inputs: n - dimensionality of the feature vectors for the kernel. 
must be a power of two and be divisible by d.  If not then can
pad the matrix with zeros but for simplicity assume this condition 
is always met.
Output: V'''
S = np.zeros(shape=(d,d))
G = np.zeros_like(S)
B = np.zeros_like(S)
H = hadamard(d)
Pi = np.eye(d)
np.random.shuffle(Pi) # Permutation matrix

# Construct the simple matrices
np.fill_diagonal(B, 2*np.random.randint(low=0,high=2,size=(d,1)).flatten() - 1)
np.fill_diagonal(G, np.random.randn(G.shape[0],1))  # May want to change standard normal to arbitrary which will affect the scaling for V
np.fill_diagonal(S, np.linalg.norm(G,'fro')**(-0.5))
#print('Shapes of B {}, S {}, G {}, H{}, Pi {}'.format(B.shape, S.shape, G.shape, H.shape, Pi.shape))

V = d**(-0.5)*S.dot(H).dot(G).dot(Pi).dot(H).dot(B)

返回V

def fastfood_feature_map(X, n):
'''Given a matrix X of data compute the fastfood transformation and feature mapping.
Input: X data of dimension d by m, n = the number of nonlinear basis functions to choose (power of 2)
Outputs: Phi - matrix of random features for fastfood kernel approximation.
Usage: Phi must be transposed for computation in the kernel ridge regression.
   i.e solve ||Phi.T * w - b || + regulariser
   Comments: This only uses a standard normal distribution but this could
   be altered with different hyperparameters.'''
d,m = A.shape
V = fastfood_product(d)
Phi = n**(-0.5)*np.exp(1j*np.dot(V, X))
return Phi

我认为上述内容需要导入numpy as np和from linalg import hadamard。

定义要在approximate_kernel.py中使用的新内核

0 个答案: