Question

我想在多类分类中测试针对lightgbm的自定义目标函数。我已指定参数“ num_class = 3”。但是，出现错误：“

对于非多班训练，班数必须为1”

我正在使用python 3.6和lightgbm 0.2版

# iris data
from sklearn import datasets
import lightgbm as lgb
import numpy as np

iris = datasets.load_iris()
X = iris['data']
y = iris['target']

# construct train-test
num_train = int(X.shape[0] / 3 * 2)
idx = np.random.permutation(X.shape[0])

x_train = X[idx[:num_train]]
x_test = X[idx[num_train:]]
y_train = y[idx[:num_train]]
y_test = y[idx[num_train:]]
# softmax function
def softmax(x):
    '''
    input x: an np.array of n_sample * n_class
    return : an np.array of n_sample * n_class (probabilities)
    '''
    x = np.where(x>100, 100, x)
    x = np.exp(x)
    return x / np.reshape(np.sum(x, 1), [x.shape[0], 1])
# objective function    
def objective(y_true, y_pred):
    '''
    input: 
        y_true: np.array of size (n_sample,)
        y_pred: np.array of size (n_sample, n_class)
    '''
    y_pred = softmax(y_pred) 
    temp = np.zeros_like(y_pred)
    temp[range(y_pred.shape[0]), y_true] = 1   
    gradient = y_pred - temp
    hessian = y_pred * (1 - y_pred)  
    return [gradient, hessian]
# lightgbm model
model = lgb.LGBMClassifier(n_estimators=10000,
                           num_classes = 3,
                           objective = objective,
                           nthread=4)
model.fit(x_train, y_train, 
          eval_metric = 'multi_logloss',
          eval_set = [(x_test, y_test), (x_train, y_train)],
          eval_names = ['valid', 'train'], 
          early_stopping_rounds = 200, verbose = 100)

Answer 1

让我回答我自己的问题。

目标函数中的参数应为：

y_true of size [n_sample, ]
y_pred of size [n_sample * n_class, ] instead of [n_sample, n_class]

更具体地说，y_pred应该像

y_pred = [first_class, first_class,..., second_class, second_class,..., third_class, third_class,...]

此外，应按相同的方式对渐变和粗麻布进行分组。

def objective(y_true, y_pred):
    '''
    input: 
        y_true: np.array of size [n_sample,]
        y_pred: np.array of size [n_sample * n_class, ]
    return:
        gradient and hessian should have exactly the same form of y_pred
    '''
    y_pred = np.reshape(y_pred, [num_train, 3], order = 'F')
    y_pred = softmax(y_pred)

    temp = np.zeros_like(y_pred)
    temp[range(y_pred.shape[0]), y_true] = 1

    gradient = y_pred - temp
    hessian = y_pred * (1 - y_pred)

    return [gradient.ravel(order = 'F'), hessian.ravel(order = 'F')]

使用自定义目标函数进行多类分类时如何为lightgbm设置参数？

1 个答案: