使用相机的透视校正获取图像（在平面上对齐）（2D图像）

时间：2019-01-03 03:11:02

标签： c++ opencv homography perspectivecamera

我想从相机中获取原始图像。

这是我的相机获得的图像。我想要的图像是紫色矩形。 This is my input

我要裁剪紫色矩形并更正该前景。这是我期望得到的图像。 This is what I want to get

图像尺寸未知。它可以宽也可以高。

如何在OpenCV中执行此操作？有提示，指南吗？请注意，对于每个标记，我已经有了每个标记角的坐标。（此信息可能会有所帮助）

编辑。一些进展。

我了解到我需要的功能是getPerspectiveTransform和warpPerspective。

我同时使用这两种方法。

    if (ids.size() == 4)
    {
        array<Point2f, 4> srcCorners;           // corner that we want 
        array<Point2f, 4> srcCornersSmall; 
        array<Point2f, 4> dstCorners;           // destination corner   

        //id  8 14 18 47
        for (size_t i = 0; i < ids.size(); i++)
        {
            // first corner
            if (ids[i] == 8)
            {
                srcCorners[0] = corners[i][0];      // get the first point
                srcCornersSmall[0] = corners[i][2];
            }
            // second corner
            else if (ids[i] == 14)
            {
                srcCorners[1] = corners[i][1];      // get the second point
                srcCornersSmall[1] = corners[i][3];
            }
            // third corner
            else if (ids[i] == 18)
            {
                srcCorners[2] = corners[i][2];      // get the thirt point
                srcCornersSmall[2] = corners[i][0];
            }
            // fourth corner
            else if (ids[i] == 47)
            {
                srcCorners[3] = corners[i][3];      // get the fourth point
                srcCornersSmall[3] = corners[i][1];
            }
        }

        dstCorners[0] = Point2f(0.0f, 0.0f);
        dstCorners[1] = Point2f(256.0f, 0.0f);
        dstCorners[2] = Point2f(256.0f, 256.0f);
        dstCorners[3] = Point2f(0.0f, 256.0f);

        // get perspectivetransform
        Mat M = getPerspectiveTransform(srcCorners, dstCorners);

        // warp perspective
        Mat dst;
        Size dsize = Size(cvRound(dstCorners[2].x), cvRound(dstCorners[2].y));
        warpPerspective(imageCopy, dst, M, dsize);

        // show
        imshow("perspective transformed", dst);

    }

虽然我几乎得到了想要的图像，但是图像的宽度/高度比不是正确的。

这是我得到的输出。

enter image description here

如何校正宽度高度比？

1 个答案:

答案 0 :(得分：0)

最后知道了。 想法是在黑色图像上将标记画为白色框。然后裁剪所需的图像并将其绘制为新图像。由于未知新图像的正确尺寸，因此我们将尺寸设置为正方形。新图像应为黑色图像，角落处带有白色框。从（0,0）开始，我们穿过图像并检查像素值。像素值应为白色。如果像素值为黑色，则我们在白色框之外。沿x和y追溯像素值，因为白框可能很高或很宽。找到白框的右下角后，便有了白框的大小。将此白色框重新缩放为正方形。使用相同的功能重新缩放图像。

这是相机拍摄的图像

enter image description here

在黑色图像中将标记绘制为白色框。

enter image description here

裁剪并扭曲成正方形。

enter image description here

获取左上角白框的宽度和高度。 一旦有了缩放功能，就可以应用它。

enter image description here

如果有兴趣的人，这里是代码。

// Get3dRectFrom2d.cpp : This file contains the 'main' function. Program execution begins and ends there.
//

#include "pch.h"
#include <iostream>
#include <opencv2/opencv.hpp>
#include <opencv2/aruco.hpp>

#define CAMERA_WINDOW "Simple ArUco"

using namespace std;
using namespace cv;

static bool readCameraParameters(string filename, Mat &camMatrix, Mat &distCoeffs) {
    FileStorage fs(filename, FileStorage::READ);
    if (!fs.isOpened())
        return false;
    fs["camera_matrix"] >> camMatrix;
    fs["distortion_coefficients"] >> distCoeffs;
    return true;
}

int main()
{
    Mat camMatrix, distCoeffs;
    string cameraSettings = "camera.txt";
    bool estimatePose = false;
    bool showRejected = true;
    if (readCameraParameters(cameraSettings, camMatrix, distCoeffs))
    {
        estimatePose = true;
    }
    Ptr<aruco::Dictionary> dictionary =
        aruco::getPredefinedDictionary(aruco::PREDEFINED_DICTIONARY_NAME(aruco::DICT_4X4_50));
    Ptr<aruco::DetectorParameters> detectorParams = aruco::DetectorParameters::create();
    float markerLength = 3.75f;
    float markerSeparation = 0.5f;
    double totalTime = 0;
    int totalIterations = 0;
    VideoCapture inputVideo(0);

    if (!inputVideo.isOpened())
    {
        cout << "cannot open camera";
    }

    double prevW = -1, prevH = -1;
    double increment = 0.1;

    while (inputVideo.grab())
    {
        Mat image, imageCopy;
        inputVideo.retrieve(image);

        double tick = (double)getTickCount();

        vector< int > ids;
        vector< vector< Point2f > > corners, rejected;
        vector< Vec3d > rvecs, tvecs;

        // detect markers and estimate pose
        aruco::detectMarkers(image, dictionary, corners, ids, detectorParams, rejected);
        if (estimatePose && ids.size() > 0)
            aruco::estimatePoseSingleMarkers(corners, markerLength, camMatrix, distCoeffs, rvecs,
                tvecs);

        double currentTime = ((double)getTickCount() - tick) / getTickFrequency();
        totalTime += currentTime;
        totalIterations++;
        if (totalIterations % 30 == 0) {
            cout << "Detection Time = " << currentTime * 1000 << " ms "
                << "(Mean = " << 1000 * totalTime / double(totalIterations) << " ms)" << endl;
        }

        // draw results
        image.copyTo(imageCopy);
        if (ids.size() > 0) {
            aruco::drawDetectedMarkers(imageCopy, corners, ids);

            if (estimatePose) {
                for (unsigned int i = 0; i < ids.size(); i++)
                    aruco::drawAxis(imageCopy, camMatrix, distCoeffs, rvecs[i], tvecs[i],
                        markerLength * 0.5f);
            }
        }

        if (ids.size() == 4)
        {
            if (true)
            {
                // process the image
                array<Point2f, 4> srcCorners;           // corner that we want 
                array<Point2f, 4> dstCorners;           // destination corner   
                vector<Point> marker0;          // marker corner
                vector<Point> marker1;          // marker corner
                vector<Point> marker2;          // marker corner
                vector<Point> marker3;          // marker corner

                //id  8 14 18 47
                for (size_t i = 0; i < ids.size(); i++)
                {
                    // first corner
                    if (ids[i] == 8)
                    {
                        srcCorners[0] = corners[i][0];      // get the first point
                        //srcCornersSmall[0] = corners[i][2];

                        marker0.push_back(corners[i][0]);
                        marker0.push_back(corners[i][1]);
                        marker0.push_back(corners[i][2]);
                        marker0.push_back(corners[i][3]);
                    }
                    // second corner
                    else if (ids[i] == 14)
                    {
                        srcCorners[1] = corners[i][1];      // get the second point
                        //srcCornersSmall[1] = corners[i][3];

                        marker1.push_back(corners[i][0]);
                        marker1.push_back(corners[i][1]);
                        marker1.push_back(corners[i][2]);
                        marker1.push_back(corners[i][3]);
                    }
                    // third corner
                    else if (ids[i] == 18)
                    {
                        srcCorners[2] = corners[i][2];      // get the thirt point
                        //srcCornersSmall[2] = corners[i][0];

                        marker2.push_back(corners[i][0]);
                        marker2.push_back(corners[i][1]);
                        marker2.push_back(corners[i][2]);
                        marker2.push_back(corners[i][3]);
                    }
                    // fourth corner
                    else if (ids[i] == 47)
                    {
                        srcCorners[3] = corners[i][3];      // get the fourth point
                        //srcCornersSmall[3] = corners[i][1];

                        marker3.push_back(corners[i][0]);
                        marker3.push_back(corners[i][1]);
                        marker3.push_back(corners[i][2]);
                        marker3.push_back(corners[i][3]);
                    }
                }

                // create a black image with the same size of cam image
                Mat mask = Mat::zeros(imageCopy.size(), CV_8UC1);
                Mat dstImage = Mat::zeros(imageCopy.size(), CV_8UC1);

                // draw white fill on marker corners
                {
                    int num = (int)marker0.size();
                    if (num != 0)
                    {
                        const Point * pt4 = &(marker0[0]);
                        fillPoly(mask, &pt4, &num, 1, Scalar(255, 255, 255), 8);
                    }
                }
                {
                    int num = (int)marker1.size();
                    if (num != 0)
                    {
                        const Point * pt4 = &(marker1[0]);
                        fillPoly(mask, &pt4, &num, 1, Scalar(255, 255, 255), 8);
                    }
                }
                {
                    int num = (int)marker2.size();
                    if (num != 0)
                    {
                        const Point * pt4 = &(marker2[0]);
                        fillPoly(mask, &pt4, &num, 1, Scalar(255, 255, 255), 8);
                    }
                }
                {
                    int num = (int)marker3.size();
                    if (num != 0)
                    {

                        const Point * pt4 = &(marker3[0]);
                        fillPoly(mask, &pt4, &num, 1, Scalar(255, 255, 255), 8);
                    }
                }

                // draw the mask
                imshow("black white lines", mask);

                // we dont have the correct size/aspect ratio
                double width = 256.0f, height = 256.0f;
                dstCorners[0] = Point2f(0.0f, 0.0f);
                dstCorners[1] = Point2f(width, 0.0f);
                dstCorners[2] = Point2f(width, height);
                dstCorners[3] = Point2f(0.0f, height);

                // get perspectivetransform
                Mat M = getPerspectiveTransform(srcCorners, dstCorners);

                // warp perspective
                Mat dst;
                Size dsize = Size(cvRound(dstCorners[2].x), cvRound(dstCorners[2].y));
                warpPerspective(mask, dst, M, dsize);

                // show warped image
                imshow("perspective transformed", dst);

                // get width and length of the first marker
                // start from (0,0) and cross 
                int cx = 0, cy = 0; // track our current coordinate
                Scalar v, vx, vy; // pixel value at coordinate
                bool cont = true;
                while (cont)
                {
                    v = dst.at<uchar>(cx, cy); // get pixel value at current coordinate
                    if (cx > 1 && cy > 1) 
                    {
                        vx = dst.at<uchar>(cx - 1, cy);
                        vy = dst.at<uchar>(cx, cy - 1);
                    }

                    // if pixel not black, continue crossing
                    if ((int)v.val[0] != 0)
                    {
                        cx++;
                        cy++;
                    }

                    // current pixel is black
                    // if previous y pixel is not black, means that we need to walk the pixel right
                    else if ((int)((Scalar)dst.at<uchar>(cx, cy - 1)).val[0] != 0)
                    {
                        cx = cx + 1;
                    }
                    // if previous x pixel is not black, means that we need to walk the pixel down
                    else if ((int)((Scalar)dst.at<uchar>(cx - 1, cy)).val[0] != 0)
                    {
                        cy = cy + 1;
                    }

                    // the rest is the same with previous 2, only with higher previous pixel to check
                    // need to do this because sometimes pixels is jagged
                    else if ((int)((Scalar)dst.at<uchar>(cx, cy - 2)).val[0] != 0)
                    {
                        cx = cx + 1;
                    }
                    else if ((int)((Scalar)dst.at<uchar>(cx - 2, cy)).val[0] != 0)
                    {
                        cy = cy + 1;
                    }

                    else if ((int)((Scalar)dst.at<uchar>(cx, cy - 3)).val[0] != 0)
                    {
                        cx = cx + 1;
                    }
                    else if ((int)((Scalar)dst.at<uchar>(cx - 3, cy)).val[0] != 0)
                    {
                        cy = cy + 1;
                    }

                    else if ((int)((Scalar)dst.at<uchar>(cx, cy - 4)).val[0] != 0)
                    {
                        cx = cx + 1;
                    }
                    else if ((int)((Scalar)dst.at<uchar>(cx - 4, cy)).val[0] != 0)
                    {
                        cy = cy + 1;
                    }

                    else if ((int)((Scalar)dst.at<uchar>(cx, cy - 5)).val[0] != 0)
                    {
                        cx = cx + 1;
                    }
                    else if ((int)((Scalar)dst.at<uchar>(cx - 5, cy)).val[0] != 0)
                    {
                        cy = cy + 1;
                    }

                    else
                    {
                        cx = cx - 1;
                        cy = cy - 1;
                        cont = false;
                    }

                    // reached the end of the picture
                    if (cx >= dst.cols)
                    {
                        cont = false;
                    }
                    else if (cy >= dst.rows)
                    {
                        cont = false;
                    }
                }

                if (cx == cy)
                {
                    //we have perfect square
                }
                if (cx > cy)
                {
                    // wide
                    width = (height * ((double)cx / (double)cy));
                }
                else
                {
                    // tall
                    height = (width * ((double)cy / (double)cx));
                }

                // we dont want the size varied too much every frame, 
                // so limits the increment or decrement for every frame
                // initialize first usage
                if (prevW<0)
                {
                    prevW = width;
                }
                if (prevH<0)
                {
                    prevH = height;
                }

                if (width > prevW + increment)
                {
                    width = prevW + increment;
                }
                else if (width < prevW - increment)
                {
                    width = prevW - increment;
                }
                prevW = width;

                if (height > prevH + increment)
                {
                    height = prevH + increment;
                }
                else if (height < prevH - increment)
                {
                    height = prevH - increment;
                }
                prevH = height;

                // show resized image
                Size s(width, height);
                Mat resized;
                resize(dst, resized, s);
                imshow("resized", resized);
            }
        }

        if (showRejected && rejected.size() > 0)
            aruco::drawDetectedMarkers(imageCopy, rejected, noArray(), Scalar(100, 0, 255));

        imshow("out", imageCopy);
        if (waitKey(1) == 27) {
            break;
        }
    }
    cout << "Hello World!\n";
    cin.ignore();
    return 0;
}

我对数学解决方案更感兴趣，但就目前而言，就足够了。如果你们知道更好的方法（更快），请告诉我。

相关问题
最新问题