Question

我正在尝试使用OpenCV Java API将两个图像拼接在一起。但是，我得到错误的输出，我无法解决问题。我使用以下步骤：检测功能 2.提取功能 3.匹配功能。找到单应性 5.找到透视变换 6.经线观点 7.将2幅图像“拼接”成合成图像。

但某处我出错了。我认为这是我梳理2张图片的方式，但我不确定。我在2张图片之间获得了214个很好的特征匹配，但无法拼接它们？

public class ImageStitching {

static Mat image1;
static Mat image2;

static FeatureDetector fd;
static DescriptorExtractor fe;
static DescriptorMatcher fm;

public static void initialise(){
    fd = FeatureDetector.create(FeatureDetector.BRISK); 
    fe = DescriptorExtractor.create(DescriptorExtractor.SURF); 
    fm = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE);

    //images
    image1 = Highgui.imread("room2.jpg");
    image2 = Highgui.imread("room3.jpg");

    //structures for the keypoints from the 2 images
    MatOfKeyPoint keypoints1 = new MatOfKeyPoint();
    MatOfKeyPoint keypoints2 = new MatOfKeyPoint();

    //structures for the computed descriptors
    Mat descriptors1 = new Mat();
    Mat descriptors2 = new Mat();

    //structure for the matches
    MatOfDMatch matches = new MatOfDMatch();

    //getting the keypoints
    fd.detect(image1, keypoints1);
    fd.detect(image1, keypoints2);

    //getting the descriptors from the keypoints
    fe.compute(image1, keypoints1, descriptors1);
    fe.compute(image2,keypoints2,descriptors2);

    //getting the matches the 2 sets of descriptors 
    fm.match(descriptors2,descriptors1, matches);

    //turn the matches to a list
    List<DMatch> matchesList = matches.toList();

    Double maxDist = 0.0; //keep track of max distance from the matches
    Double minDist = 100.0; //keep track of min distance from the matches

    //calculate max & min distances between keypoints
    for(int i=0; i<keypoints1.rows();i++){
        Double dist = (double) matchesList.get(i).distance;
        if (dist<minDist) minDist = dist;
        if(dist>maxDist) maxDist=dist;
    }

    System.out.println("max dist: " + maxDist );
    System.out.println("min dist: " + minDist);

    //structure for the good matches
    LinkedList<DMatch> goodMatches = new LinkedList<DMatch>();

    //use only the good matches (i.e. whose distance is less than 3*min_dist)
    for(int i=0;i<descriptors1.rows();i++){
        if(matchesList.get(i).distance<3*minDist){
            goodMatches.addLast(matchesList.get(i));
        }
    }

    //structures to hold points of the good matches (coordinates)
    LinkedList<Point> objList = new LinkedList<Point>(); // image1
    LinkedList<Point> sceneList = new LinkedList<Point>(); //image 2

    List<KeyPoint> keypoints_objectList = keypoints1.toList();
    List<KeyPoint> keypoints_sceneList = keypoints2.toList();

    //putting the points of the good matches into above structures
    for(int i = 0; i<goodMatches.size(); i++){
        objList.addLast(keypoints_objectList.get(goodMatches.get(i).queryIdx).pt);
        sceneList.addLast(keypoints_sceneList.get(goodMatches.get(i).trainIdx).pt);
    }

    System.out.println("\nNum. of good matches" +goodMatches.size());

    MatOfDMatch gm = new MatOfDMatch();
    gm.fromList(goodMatches);

    //converting the points into the appropriate data structure
    MatOfPoint2f obj = new MatOfPoint2f();
    obj.fromList(objList);

    MatOfPoint2f scene = new MatOfPoint2f();
    scene.fromList(sceneList);

    //finding the homography matrix
    Mat H = Calib3d.findHomography(obj, scene);

    //LinkedList<Point> cornerList = new LinkedList<Point>();
    Mat obj_corners = new Mat(4,1,CvType.CV_32FC2);
    Mat scene_corners = new Mat(4,1,CvType.CV_32FC2);

    obj_corners.put(0,0, new double[]{0,0});
    obj_corners.put(0,0, new double[]{image1.cols(),0});
    obj_corners.put(0,0,new double[]{image1.cols(),image1.rows()});
    obj_corners.put(0,0,new double[]{0,image1.rows()});

    Core.perspectiveTransform(obj_corners, scene_corners, H);

    //structure to hold the result of the homography matrix
    Mat result = new Mat();

    //size of the new image - i.e. image 1 + image 2
    Size s = new Size(image1.cols()+image2.cols(),image1.rows());

    //using the homography matrix to warp the two images
    Imgproc.warpPerspective(image1, result, H, s);
    int i = image1.cols();
    Mat m = new Mat(result,new Rect(i,0,image2.cols(), image2.rows()));

    image2.copyTo(m);

    Mat img_mat = new Mat();

    Features2d.drawMatches(image1, keypoints1, image2, keypoints2, gm, img_mat, new Scalar(254,0,0),new Scalar(254,0,0) , new MatOfByte(), 2);

    //creating the output file
    boolean imageStitched = Highgui.imwrite("imageStitched.jpg",result);
    boolean imageMatched = Highgui.imwrite("imageMatched.jpg",img_mat);
}


public static void main(String args[]){
    System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
    initialise();
}

由于声望点，我无法嵌入图片或发布超过2个链接？所以我把错误拼接的图像与显示两幅图像之间匹配特征的图像联系起来（以了解问题）：

错误的拼接图片：http://oi61.tinypic.com/11ac01c.jpg 检测到的功能：http://oi57.tinypic.com/29m3wif.jpg

Answer 1

似乎你有很多异常值会使单应性的估计不正确。因此，您可以使用递归拒绝这些异常值的RANSAC方法。

不需要太多努力，只需在findHomography函数中使用第三个参数：

Mat H = Calib3d.findHomography(obj, scene, CV_RANSAC);

修改

然后尝试确保给检测器的图像是8位灰度图像，如上所述here

Answer 2

您发布的“错误拼接的图像”看起来像是有一个糟糕的条件H矩阵。除了+托维护之外的建议，运行：

cv::determinant(H) > 0.01

检查你的H矩阵是否“可用”。如果矩阵条件不好，您将获得正在显示的效果。

您正在绘制2x2画布大小，如果是这种情况，您将看不到大量拼接配置，即图像B左侧的图像A没有，但不是。尝试使用以下代码段将输出绘制到3x3画布大小：

  // Use the Homography Matrix to warp the images, but offset it to the
  // center of the output canvas. Careful to pre-multiply, not post-multiply.
  cv::Mat Offset = (cv::Mat_<double>(3,3) << 1, 0, 
                    width, 0, 1, height, 0, 0, 1);
  H = Offset * H;

  cv::Mat result;
  cv::warpPerspective(mat_l,
                      result,
                      H,
                      cv::Size(3*width, 3*height));
  // Copy the reference image to the center of the 3x3 output canvas.
  cv::Mat roi = result.colRange(width,2*width).rowRange(height,2*height);
  mat_r.copyTo(roi);

其中width和height是输入图像的图像，据推测它们都具有相同的大小。请注意，此变形假定mat_l未更改（平坦）和mat_r变形以拼接它。

拼接2张图片（OpenCV）

2 个答案: