Yolo在对象检测中使用的Indexer上JavaCV 1.5和1.4.4之间的区别

时间：2019-06-08 22:44:40

标签： java opencv javacv yolo

我的环境：

Ubuntu 18.04 LTS 64位

Java：OpenJDK8 + Groovy 2.5.6

JavaCV版本：1.4.4与1.5

Yolo版本：Yolo v3 tiny

我在JavaCV中使用open_dnn通过Yolo检测对象。 我的代码在JavaCV 1.4.4上运行良好。并且可以毫无问题地检测物体。 但是使用相同的代码（已调整导入包以匹配JavaCV 1.5），相同的yolo配置，相同的权重，相同的测试图片，它什么也检测不到。我试了几次，结果是一样的。

我只列出相关代码并记录如下： 对于JavaCV 1.4.4，代码如下：

import groovy.util.logging.Slf4j
import org.bytedeco.javacpp.FloatPointer
import org.bytedeco.javacpp.IntPointer
import org.bytedeco.javacpp.indexer.FloatIndexer
import org.bytedeco.opencv.global.opencv_core
import org.bytedeco.opencv.global.opencv_dnn
import org.bytedeco.opencv.global.opencv_imgproc
import org.bytedeco.javacv.Java2DFrameUtils

import  org.bytedeco.opencv.opencv_dnn.Net
import  org.bytedeco.opencv.opencv_core.Rect
import  org.bytedeco.opencv.opencv_core.RectVector
import  org.bytedeco.opencv.opencv_core.Scalar
import  org.bytedeco.opencv.opencv_core.Size
import  org.bytedeco.opencv.opencv_core.IplImage
import  org.bytedeco.opencv.opencv_core.Mat
import  org.bytedeco.opencv.opencv_core.MatVector

import org.bytedeco.javacv.Frame
import vehicle.detectedobject.DetectedVehicle

@Slf4j
class YoloV3TinyVehicleDetector  {
    private Net net
    private List<String> classNameList
    private float confThreshold = 0.5; // Confidence threshold
    private float nmsThreshold = 0.4;
    private final static Scalar scalar = new Scalar(0)
    private final static Size sz = new Size(416, 416)
    private final static Size sz1 = new Size(1280, 720)
    private final static float scale = 1F/255F
    private final static boolean swapRB = true
    private Rect[] lastBoxArray
    private boolean foundNothing = false

    public YoloV3TinyVehicleDetector(
                String moduleConfigFilename, String moduleWeightFilename,
                int backendId, int targetId,
                List<String> classNameList
    ) {
        net = opencv_dnn.readNetFromDarknet(moduleConfigFilename, moduleWeightFilename)
        net.setPreferableBackend(opencv_dnn.DNN_BACKEND_DEFAULT)
        net.setPreferableTarget(opencv_dnn.DNN_TARGET_CPU)
        this.classNameList = classNameList
    }    

    List<DetectedVehicle> detectVehicles(Mat frame) {

        Mat inputBlob = opencv_dnn.blobFromImage(frame, scale, sz,scalar, swapRB, false, opencv_core.CV_32F)

        net.setInput(inputBlob)
        MatVector outputBlobs = new MatVector()
        net.forward(outputBlobs, net.getLayerNames())

        List<Integer> classIds =[]
        List<Float> confidences = []
        List<Rect> boxes = []
        List<DetectedVehicle> detectedVehicleList = []
        FloatIndexer indexer
        for(int ii = 0; ii < outputBlobs.size(); ii++) {
            Mat detectionMat = outputBlobs.get(ii)
            if (detectionMat.empty()) {
                foundNothing = true
                log.debug("detectVehicles(): No object detected!")
                return detectedVehicleList
            }
            int rowCount = detectionMat.rows()
            log.debug("detectVehicles(): ii[${ii}]. detectionMat.cols(), rows()=${detectionMat.cols()},${rowCount}")
            if (rowCount <=0) continue
            if (rowCount > 0) {
                indexer = detectionMat.createIndexer()

            }
            for(int i = 0; i < rowCount; i++) {
                int probability_index = 5;
                int size = (int) (detectionMat.cols() * detectionMat.channels());
                float[] data = new float[size]
                indexer.get(i, 0, data)
                log.debug("detectVehicles(): i[${i}/${detectionMat.rows()}]:size:${size}, data:"+data) // <-- here is the last line of output
            ...
            ...
            ...
    }
    ...
    ...
}
以下是JavaCV 1.4.4的

日志消息。两个代码之间日志的不同部分位于输出的最后一行

> Task :TestPlayWithYoloV3.main()
detectVehicles(): image wxh=800x480
detectVehicles(): mat wxh=800x480
19/06/09 06:06:41.793 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[0]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[1]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[2]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[3]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[4]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[5]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[6]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[7]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[8]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[9]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[10]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[11]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.795 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[12]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.796 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[13]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.796 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[14]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.796 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[15]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.796 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[16]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[17]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[18]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[19]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[20]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[21]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[22]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[23]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[24]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[25]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[26]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[27]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[28]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[29]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[30]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[31]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[32]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.797 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[33]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.798 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[34]. detectionMat.cols(), rows()=-1,-1
19/06/09 06:06:41.798 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[35]. detectionMat.cols(), rows()=12,507
19/06/09 06:06:41.817 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): i[0/507]:size:12, data:[0.03504561, 0.035271123, 0.03801521, 0.20626219, 1.4282074E-27, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
19/06/09 06:06:41.818 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): i[1/507]:size:12, data:[0.043720312, 0.027087418, 0.17653388, 0.10258253, 5.213337E-34, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

对于JavaCV 1.5，代码在下面（就像我上面说的那样，除了导入的调整外没有其他变化

import groovy.util.logging.Slf4j
import org.bytedeco.javacpp.FloatPointer
import org.bytedeco.javacpp.IntPointer
import org.bytedeco.javacpp.indexer.FloatIndexer
import org.bytedeco.opencv.global.opencv_core
import org.bytedeco.opencv.global.opencv_dnn
import org.bytedeco.opencv.global.opencv_imgproc
import org.bytedeco.javacv.Java2DFrameUtils

import  org.bytedeco.opencv.opencv_dnn.Net
import  org.bytedeco.opencv.opencv_core.Rect
import  org.bytedeco.opencv.opencv_core.RectVector
import  org.bytedeco.opencv.opencv_core.Scalar
import  org.bytedeco.opencv.opencv_core.Size
import  org.bytedeco.opencv.opencv_core.IplImage
import  org.bytedeco.opencv.opencv_core.Mat
import  org.bytedeco.opencv.opencv_core.MatVector

import org.bytedeco.javacv.Frame
import vehicle.detectedobject.DetectedVehicle

import java.awt.image.BufferedImage

@Slf4j
class YoloV3TinyVehicleDetector  {
    private Net net
    private List<String> classNameList
    private float confThreshold = 0.5; // Confidence threshold
    private float nmsThreshold = 0.4;
    private final static Scalar scalar = new Scalar(0)
    private final static Size sz = new Size(416, 416)
    private final static Size sz1 = new Size(1280, 720)
    private final static float scale = 1F/255F
    private final static boolean swapRB = true
    private Rect[] lastBoxArray
    private boolean foundNothing = false

    public YoloV3TinyVehicleDetector(
            String moduleConfigFilename, String moduleWeightFilename,
            int backendId, int targetId,
            List<String> classNameList
    ) {
        net = opencv_dnn.readNetFromDarknet(moduleConfigFilename, moduleWeightFilename)
        net.setPreferableBackend(opencv_dnn.DNN_BACKEND_DEFAULT)
        net.setPreferableTarget(opencv_dnn.DNN_TARGET_CPU)
        this.classNameList = classNameList
    }


    List<DetectedVehicle> detectVehicles(Mat frame) {

        log.debug("detectVehicles(): im wxh=${frame.cols()}x${frame.rows()}")
        Mat inputBlob = opencv_dnn.blobFromImage(frame, scale, sz,scalar, swapRB, false, opencv_core.CV_32F)
        net.setInput(inputBlob)
        MatVector outputBlobs = new MatVector()
        net.forward(outputBlobs, net.getLayerNames())
        List<Integer> classIds =[]
        List<Float> confidences = []
        List<Rect> boxes = []
        List<DetectedVehicle> detectedVehicleList = []
        FloatIndexer indexer
        log.info("outputBlobs.size()=" + outputBlobs.size())
        for(int ii = 0; ii < outputBlobs.size(); ii++) {
            Mat detectionMat = outputBlobs.get(ii)
            if (detectionMat.empty()) {
                foundNothing = true
                log.debug("detectVehicles(): detectionMat is empty, so no object detected!")
                return detectedVehicleList
            }
            int rowCount = detectionMat.rows()
            log.debug("detectVehicles(): ii[${ii}]. detectionMat.cols(),rows()=${detectionMat.cols()}, ${rowCount}")
            if (rowCount <=0) continue
            if (rowCount > 0) {
                indexer = detectionMat.createIndexer()
            }
            for(int i = 0; i < rowCount; i++) {
                int probability_index = 5;
                int size = (int) (detectionMat.cols() * detectionMat.channels());
                float[] data = new float[size]
                indexer.get(i, 0, data)
                log.debug("detectVehicles(): i[${i}/${detectionMat.rows()}]:size:${size}, data:"+data) // <-- here is the last line of output
                ...
                ...
                ...
    }
}

我的JavaCV1.5检测器的日志输出：

detectVehicles(): image wxh=800x480
detectVehicles(): mat wxh=800x480
19/06/08 18:07:23.553 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): im wxh=800x480
19/06/08 18:07:23.976 INFO YoloV3TinyVehicleDetector(): outputBlobs.size()=48
19/06/08 18:07:23.979 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[0]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.979 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[1]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.979 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[2]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.979 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[3]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.979 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[4]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.979 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[5]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.980 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[6]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.980 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[7]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.980 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[8]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.980 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[9]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.980 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[10]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.980 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[11]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.980 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[12]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.980 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[13]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.981 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[14]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.981 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[15]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.981 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[16]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.981 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[17]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.981 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[18]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.981 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[19]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[20]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[21]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[22]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[23]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[24]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[25]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[26]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[27]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[28]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[29]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[30]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[31]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.982 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[32]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.983 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[33]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.983 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[34]. detectionMat.cols(),rows()=-1, -1
19/06/08 18:07:23.983 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): ii[35]. detectionMat.cols(),rows()=12, 507
19/06/08 18:07:23.997 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): i[0/507]:size:12, data:[NaN, NaN, NaN, NaN, NaN, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
19/06/08 18:07:23.997 DEBUG YoloV3TinyVehicleDetector(): detectVehicles(): i[1/507]:size:12, data:[NaN, NaN, NaN, NaN, NaN, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

但是，使用相同的算法，我只是使用opencv 4.1.0（不是JavaCV），它仍然可以工作并且可以检测对象。因此，我怀疑JavaCV 1.5会有什么不同之处。

还是我错过了JavaCV 1.5的新用法？

谢谢您的帮助。

0 个答案:

没有答案
相关问题
最新问题