Question

我已经在python中实现了Lucas Kanade算法，并使用它来跟踪使用Harris角点检测器获得的点。我现在正在足球比赛视频上测试该算法以跟踪球员。该算法在开始的几秒钟内效果很好，然后在几秒钟后（没有任何遮挡）失去了被跟踪的玩家。

我的实现在以下方法中：

   # This is the method that implements the Lucas Kanade algorithm
def calc_optical_flow(self, im1, im2, corners: list, win=5):
    assert im1.shape == im2.shape

    x_kernel = np.array([[-1., 1.], [-1., 1.]])
    y_kernel = np.array([[-1., -1.], [1., 1.]])
    t_kernel = np.array([[1., 1.], [1., 1.]])

    win_size = math.floor(win / 2)

    # Implement Lucas Kanade
    # for each point, calculate I_x, I_y, I_t
    mode = 'same'  # This ensures that the convolution returns the same shape as the images
    boundary = 'symm'
    dx = signal.convolve2d(im1, x_kernel, boundary=boundary, mode=mode)
    dy = signal.convolve2d(im1, y_kernel, boundary=boundary, mode=mode)
    dt = signal.convolve2d(im2, t_kernel, boundary=boundary, mode=mode) + signal.convolve2d(im1, -t_kernel,
                                                                                            boundary=boundary,
                                                                                            mode=mode)
    u = np.zeros(len(corners))
    v = np.zeros(len(corners))
    # within window window_size * window_size

    for index, k in enumerate(corners):
        x = k[0]
        y = k[1]

        ix = dx[x - win_size: x + win_size, y - win_size: y + win_size + 1].flatten()
        iy = dy[x - win_size: x + win_size, y - win_size: y + win_size + 1].flatten()
        it = dy[x - win_size: x + win_size, y - win_size: y + win_size + 1].flatten()

        b = np.reshape(it, (it.shape[0], 1))  # get b here. Make b a column vector.
        a = np.vstack((ix, iy)).T  # get A here. Combine ix and iy into a matrix and transpose them.
        nu = np.matmul(np.linalg.pinv(a), b)  # get velocity here. Matrix inversion requires a square matrix but
        # a isn't square.
        # This is why we use pinv.

        u[index] = nu[0]
        v[index] = nu[1]

    return u, v

我已经测试过是否在棋盘图像上返回了正确的角，并且至少对于该图像效果良好。

获得光流矢量后，我将边界矩形分别移动u和v。是否有方法可以改善结果？我的代码有问题吗？

如果您有兴趣查看完整的实现，则为：https://pastebin.com/nsX2Nzg4

如何改善lucas kanade实施的结果？

0 个答案: