编辑：

Question

我正在尝试模拟站立在标记之外的图像。到目前为止，这是我的代码。基本上，我只想旋转图像看起来与棋盘正交。

如您所见，我使用代码查找标准化方形图像和相应棋盘角之间的变换矩阵。然后我使用warpPerspective来获取您看到的图像。我知道我可以使用solvePnP中的旋转向量来通过rodrigues()获得旋转矩阵，但我不知道下一步是什么

enter image description here

def transformTheSurface(inputFrame):
    ret, frameLeft = capleft.read()
    capGray = cv2.cvtColor(frameLeft,cv2.COLOR_BGR2GRAY)
    found, corners = cv2.findChessboardCorners(capGray, (5,4), None, cv2.CALIB_CB_NORMALIZE_IMAGE + cv2.CALIB_CB_ADAPTIVE_THRESH ) #,None,cv2.CALIB_CB_FAST_CHECK)
    if (found):
        npGameFrame = pygame.surfarray.array3d(inputFrame)
        inputFrameGray = cv2.cvtColor(npGameFrame,cv2.COLOR_BGR2GRAY)
        cv2.drawChessboardCorners(frameLeft, (5,4), corners, found)
        q = corners[[0, 4, 15, 19]]
        ret, rvecs, tvecs = cv2.solvePnP(objp, corners, mtx, dist)
        ptMatrix = cv2.getPerspectiveTransform( muffinCoords, q)

        npGameFrame = cv2.flip(npGameFrame, 0)

        ptMatrixWithXRot = ptMatrix * rodRotMat[0]
        #inputFrameConv = cv2.cvtColor(npGameFrame,cv2.COLOR_BGRA2GRAY)
        transMuffin = cv2.warpPerspective(npGameFrame, ptMatrix, (640, 480)) #, muffinImg, cv2.INTER_NEAREST, cv2.BORDER_CONSTANT,  0)

编辑：

我添加了一些代码，希望创建自己的3x3转换矩阵。我使用了following reference 。这是我的代码：

#initialization happens earlier in code
muffinCoords = np.zeros((4,2), np.float32)
muffinCoords[0] = (0,0)
muffinCoords[1] = (200,0)
muffinCoords[2] = (0,200)
muffinCoords[3] = (200,200)

A1 = np.zeros((4,3), np.float32)
A1[0] = (1,0,322)
A1[1] = (0,1,203)
A1[2] = (0,0,0)
A1[3] = (0,0,1)

R = np.zeros((4,4), np.float32)
R[3,3] = 1.0

T = np.zeros((4,4), np.float32)
T[0] = (1,0,0,0)
T[1] = (0,1,0,0)
T[2] = (0,0,1,0)
T[3] = (0,0,0,1)
#end initialization

#load calib data derived using cv2.calibrateCamera, my Fx and Fy are about 800
loadedCalibFileMTX = np.load('calibDataMTX.npy')
mtx = np.zeros((3,4), np.float32)
mtx[:3,:3] = loadedCalibFileMTX

output of camera calibration

#this is new to my code, creating what I interpret as Rx*Ry*Rz
ret, rvecCalc, tvecs = cv2.solvePnP(objp, corners, loadedCalibFileMTX, dist)
rodRotMat = cv2.Rodrigues(rvecCalc)
R[:3,:3] = rodRotMat[0]

output of R

#then I create T
T[0,3] = tvecs[0]
T[1,3] = tvecs[1]
T[2,3] = tvecs[2]

output of T

# CREATING CUSTOM TRANSFORM MATRIX
# A1 -> 2d to 3d projection matrix
# R-> rotation matrix as calculated by solve PnP, or Rx * Ry * Rz
# T -> converted translation matrix, reference from site, vectors pulled from tvecs of solvPnP
# mtx -> 3d to 2d matrix
# customTransformMat = mtx * (T * (R * A1)) {this is intended calculation of following}
first = np.dot(R, A1)
second = np.dot(T, first)
finalCalc = np.dot(mtx, second)

output of finalCalc as followed by formula in comments

finalNorm = finalCalc/(finalCalc[2,2]) # to make sure that the [2,2] element is 1

output of finalNorm

transMuffin = cv2.warpPerspective(npGameFrame, finalNorm, (640, 480), None, cv2.INTER_NEAREST, cv2.BORDER_CONSTANT,  0)
#transMuffin is returned as undefined here, any help?

# using the cv2.getPerspectiveTransform method to find what you can find pictured at the top
ptMatrix = cv2.getPerspectiveTransform( muffinCoords, q)

output of ptMatrix

Answer 1

我终于找到了正确的方法。你可以在https://github.com/mikezucc/augmented-reality-fighter-pygame

找到代码

注意：

几乎所有的游戏代码都是由Leif Theiden编写的，并且在.py文件中指定了许可证。与计算机视觉相关的代码在states.py中。我使用游戏只是表明可以为那些希望开始使用简单计算机视觉的人做好准备。

我的代码每次调用一个新的表面（简单的PyGame for frame）在主窗口上显示时都会打开一个线程。我在那时开始一个线程，并执行一个简单的计算机视觉功能，执行以下操作：

搜索5x4棋盘的相机流帧（cv2.findChessboardCorners）
然后将找到的角落绘制到图像上
使用cv2.solvePnP，推导出近似姿势（旋转和平移向量）
然后，将描述正方形的3d点从步骤3确定的3d空间投影到2d空间。这用于将预定义的三维结构转换为可用于在二维图像上绘制图形的内容。
然而，此步骤改为找到从一组2d方形点（游戏框架的尺寸）到新发现的投影2d点（3d框架）的变换。现在你可以看到我们正在尝试的只是做一个两步转换。
然后我执行基本教程样式添加捕获的流帧和变换后的游戏帧以获得最终图像

变量：

+ from3dTransMatrix - ＆gt;投影的3d结构的点分为2d点。这些是你看到的红点

+ q - ＆gt;这是我们从

+ ptMatrix - ＆gt;最后的转换，改变游戏框架以适应投影框架

查看最顶层文件夹中的屏幕;]

享受！

opencv python - 应用rodrigues函数的旋转矩阵

编辑：

1 个答案:

注意：