Question

在对象检测算法中，非最大抑制（NMS）用于丢弃对象的额外检测结果，例如一辆车。

通常，水平边界框用于对象检测算法中，并且水平NMS的GPU实现已经存在，但是我想让GPU实现旋转边界框。

CPU实现已经完成，但是我正在努力使用CuPy软件包将CPU版本转换为GPU版本。这是我编写的代码。在代码部分之后，您可以看到错误。

我的问题是TypeError的原因是什么：列表索引必须是整数或切片，而不是cupy.core.core.ndarray？

    from shapely.geometry import Polygon as shpoly
    import time

    #### CPU implementation
    import numpy as np   

    def polygon_iou(poly1, poly2):
      """
      Intersection over union between two shapely polygons.
      """
      if not poly1.intersects(poly2): # this test is fast and can accelerate calculation
        iou = 0
      else:
        try:
          inter_area = poly1.intersection(poly2).area
          union_area = poly1.area + poly2.area - inter_area
          iou = float(inter_area) / float(union_area)
        except shapely.geos.TopologicalError:
          warnings.warn("'shapely.geos.TopologicalError occured, iou set to 0'", UserWarning)
          iou = 0
        except ZeroDivisionError:
          iou = 0
      return iou

    def polygon_from_array(poly_):
      """
      Create a shapely polygon object from gt or dt line.
      """
      polygon_points = np.array(poly_).reshape(4, 2)
      polygon = shpoly(polygon_points).convex_hull
      return polygon

    def nms(dets, thresh):
        scores = dets[:, 8]
        order = scores.argsort()[::-1]
        polys = []
        areas = []
        for i in range(len(dets)):
            tm_polygon = polygon_from_array(dets[i,:8])
            polys.append(tm_polygon)
        keep = []
        while order.size > 0:
            ovr = []
            i = order[0]
            keep.append(i)
            for j in range(order.size - 1):
                iou = polygon_iou(polys[i], polys[order[j + 1]])
                ovr.append(iou)
            ovr = np.array(ovr)
            inds = np.where(ovr <= thresh)[0]
            order = order[inds + 1]
        return keep


    #### GPU implementation
    import cupy as cp  

    def polygon_iou_gpu(poly1, poly2):
      """
      Intersection over union between two shapely polygons.
      """
      if not poly1.intersects(poly2): # this test is fast and can accelerate calculation
        iou = 0
      else:
        try:
          inter_area = poly1.intersection(poly2).area
          union_area = poly1.area + poly2.area - inter_area
          iou = float(inter_area) / float(union_area)
        except shapely.geos.TopologicalError:
          warnings.warn("'shapely.geos.TopologicalError occured, iou set to 0'", UserWarning)
          iou = 0
        except ZeroDivisionError:
          iou = 0
      return iou

    def polygon_from_array_gpu(poly_):
      """
      Create a shapely polygon object from gt or dt line.
      """
      polygon_points = cp.array(poly_).reshape(4, 2)
      polygon = shpoly(polygon_points).convex_hull
      return polygon

    def nms_gpu(dets, thresh):
        scores = dets[:, 8]
        order = scores.argsort()[::-1]
        polys = []
        areas = []
        for i in range(len(dets)):
            tm_polygon = polygon_from_array_gpu(dets[i,:8])
            polys.append(tm_polygon)
        keep = []
        while order.size > 0:
            ovr = []
            i = order[0]
            keep.append(i)
            for j in range(order.size - 1):   
                iou = polygon_iou_gpu(polys[i], polys[order[j + 1]])
                ovr.append(iou)
            ovr = np.array(ovr)
            inds = np.where(ovr <= thresh)[0]
            order = order[inds + 1]
        return keep


    if __name__ == '__main__':
        import random
        boxes = np.random.randint(0,100,(1000,8))
        scores = np.random.rand(1000, 1)
        dets = np.hstack((boxes, scores[:])).astype(np.float32)


        thresh = 0.1
        start = time.time()
        keep = nms(dets, thresh)
        print("CPU implementation took: {}".format(time.time() - start))

        cp.cuda.Device(1)
        dets_gpu = cp.array(dets)
        start = time.time()
        keep = nms_gpu(dets_gpu, thresh)
        print("GPU implementation took: {}".format(time.time() - start))

错误是

CPU实施时间：0.3672311305999756

回溯（最近通话最近一次）：


中的文件“ nms_rotated.py”，第117行
keep = nms_gpu(dets_gpu, thresh)
文件“ nms_rotated.py”，第97行，位于nms_gpu中
iou = polygon_iou_gpu(polys[i], polys[order[j + 1]])
TypeError：列表索引必须是整数或切片，而不是cupy.core.core.ndarray

更新：13.02.2019 我尝试了@Yuki Hashimoto的答案

通过将iou = polygon_iou_gpu(polys[i], polys[order[j + 1]])替换为iou = polygon_iou_gpu(polys[i.get()], polys[order[j + 1].get()])。它不会引发任何错误，但是GPU版本的速度比CPU版本慢了好几倍。

使用100000次随机检测：

      CPU implementation took: 47.125494956970215
      GPU implementation took: 142.08464860916138

Answer 1

简而言之：使用PFN的官方non-maximum suppression。

详细信息：使用cp.where，它返回符合某些条件的list对象。

不建议使用corochann的答案，因为polys是一个列表，并且list也不应由np.ndarray分割。（并且不建议注入其他依赖项...）

>>> polys[order.get()]  # get method returns np.ndarray
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: only integer scalar arrays can be converted to a scalar index
>>> polys[order[j + 1].get()]
### some result in some case, but this may fails depending on your env.###

Answer 2

[UPDATE 2019/2/13]

请参考@ yuki-hashimoto的答案，这更合适。

如错误消息中所述

TypeError：列表索引必须是整数或切片，而不是cupy.core.core.ndarray

我猜order是cupy数组吗？在那种情况下，polys[order[j + 1]]使用索引order[j+1]作为cupy数组，这可能会导致问题。如何尝试通过cuda.to_cpu(array)方法将它们转换为numpy数组？

from chainer import cuda
iou = polygon_iou_gpu(polys[i], polys[cuda.to_cpu(order[j + 1])])

TypeError：列表索引必须是整数或切片，而不是cupy.core.core.ndarray

2 个答案: