在屏幕上搜索矩形并与图像进行比较？ Imagesearching

Question

我想知道如何在屏幕上搜索设定的矩形并将其与我指定的图像进行比较以查看它是否匹配？

让我们说它可以搜索x1 y1到x2 y2并与图像进行比较？并返回布尔值？

我知道Auto-it具有类似的功能：http://www.autohotkey.com/docs/commands/ImageSearch.htm

有没有人这样做，他们可以参考？我正在使用vb.net。

编辑：Abdias，我把你的代码放到了一个类中，我这样称呼它：

     Dim bm As Bitmap = Bitmap.FromFile(Label1.Text)
    Dim bm2 As Bitmap = Bitmap.FromFile(Label2.Text)
    Dim pnt As Point = ImageFinder.Contains(bm, bm2)
    If pnt <> Nothing Then
        MessageBox.Show("Possible match found at " & pnt.X.ToString() & " " & pnt.Y.ToString())
    Else
        MessageBox.Show("No match.")
    End If

似乎我尝试的每一组图像都没有任何意义。即使他们100％包含彼此。我拍摄了一张照片并将其裁剪了几张，并且仍然没有返回匹配。我确保来源更大。我尝试将一些图像保存为油漆中的24位jpg，但仍然没有。

以下是两张示例图片。

Answer 1

我做了这个功能，可以看到图像是否存在于更大的图像中。它被写为扩展，但可以很容易地修改为正常功能以及支持区域。

使用它：

• 将主要图片作为Bitmap
加载到标准bmp中
• 加载图片以查找bmpSearch

然后致电：

Dim pt as Point = bmp.Contains(bmpSearch)
If pt <> Nothing Then
    '... image found at pt
End If

扩展程序的代码（优化空间，但在本网站上另写为20分钟练习）：

'
'-- Extension for Bitmap
'
<Extension()>
Public Function Contains(src As Bitmap, ByRef bmp As Bitmap) As Point
    '
    '-- Some logic pre-checks
    '
    If src Is Nothing OrElse bmp Is Nothing Then Return Nothing

    If src.Width = bmp.Width AndAlso src.Height = bmp.Height Then
        If src.GetPixel(0, 0) = bmp.GetPixel(0, 0) Then
            Return New Point(0, 0)
        Else
            Return Nothing
        End If
    ElseIf src.Width < bmp.Width OrElse src.Height < bmp.Height Then
        Return Nothing
    End If
    '
    '-- Prepare optimizations
    '
    Dim sr As New Rectangle(0, 0, src.Width, src.Height)
    Dim br As New Rectangle(0, 0, bmp.Width, bmp.Height)

    Dim srcLock As BitmapData = src.LockBits(sr, Imaging.ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb)
    Dim bmpLock As BitmapData = bmp.LockBits(br, Imaging.ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb)

    Dim sStride As Integer = srcLock.Stride
    Dim bStride As Integer = bmpLock.Stride

    Dim srcSz As Integer = sStride * src.Height
    Dim bmpSz As Integer = bStride * bmp.Height

    Dim srcBuff(srcSz) As Byte
    Dim bmpBuff(bmpSz) As Byte

    Marshal.Copy(srcLock.Scan0, srcBuff, 0, srcSz)
    Marshal.Copy(bmpLock.Scan0, bmpBuff, 0, bmpSz)

    ' we don't need to lock the image anymore as we have a local copy
    bmp.UnlockBits(bmpLock)
    src.UnlockBits(srcLock)

    Dim x, y, x2, y2, sx, sy, bx, by, sw, sh, bw, bh As Integer
    Dim r, g, b As Byte

    Dim p As Point = Nothing

    bw = bmp.Width
    bh = bmp.Height

    sw = src.Width - bw      ' limit scan to only what we need. the extra corner
    sh = src.Height - bh     ' point we need is taken care of in the loop itself.

    bx = 0 : by = 0
    '
    '-- Scan source for bitmap
    '
    For y = 0 To sh
        sy = y * sStride
        For x = 0 To sw

            sx = sy + x * 3
            '
            '-- Find start point/pixel
            '
            r = srcBuff(sx + 2)
            g = srcBuff(sx + 1)
            b = srcBuff(sx)

            If r = bmpBuff(2) AndAlso g = bmpBuff(1) AndAlso b = bmpBuff(0) Then
                p = New Point(x, y)
                '
                '-- We have a pixel match, check the region
                '
                For y2 = 0 To bh - 1
                    by = y2 * bStride
                    For x2 = 0 To bw - 1
                        bx = by + x2 * 3

                        sy = (y + y2) * sStride
                        sx = sy + (x + x2) * 3

                        r = srcBuff(sx + 2)
                        g = srcBuff(sx + 1)
                        b = srcBuff(sx)

                        If Not (r = bmpBuff(bx + 2) AndAlso
                                g = bmpBuff(bx + 1) AndAlso
                                b = bmpBuff(bx)) Then
                            '
                            '-- Not matching, continue checking
                            '
                            p = Nothing
                            sy = y * sStride
                            Exit For
                        End If

                    Next
                    If p = Nothing Then Exit For
                Next
            End If 'end of region check

            If p <> Nothing Then Exit For
        Next
        If p <> Nothing Then Exit For
    Next

    bmpBuff = Nothing
    srcBuff = Nothing

    Return p

End Function

Answer 2

我玩了一下，并认为使用类似KMP算法的东西（我猜这很容易实现 - ＆gt;维基百科有一个很好的伪代码）也可能有帮助：

Imports System.Drawing.Imaging
Imports System.Runtime.InteropServices

Public Class ImageFinder

Public Shared Function Contains(Parent As Bitmap, Child As Bitmap) As Point

    If Parent Is Nothing OrElse Child Is Nothing Then Throw New ArgumentException("Narf!")
    If Parent.PixelFormat <> Imaging.PixelFormat.Format32bppArgb OrElse Child.PixelFormat <> Imaging.PixelFormat.Format32bppArgb Then Throw New ArgumentException("Narf again!")

    If Parent.Width = Child.Width AndAlso Parent.Height = Child.Height AndAlso Parent.GetPixel(0, 0) <> Child.GetPixel(0, 0) Then Return Nothing
    If Child.Width > Parent.Width OrElse Child.Height > Parent.Height Then Return Nothing

    Dim bmdParent, bmdChild As BitmapData
    Try
        ' Get bitmap data into array of int
        bmdParent = Parent.LockBits(New Rectangle(0, 0, Parent.Width, Parent.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb)
        bmdChild = Child.LockBits(New Rectangle(0, 0, Child.Width, Child.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb)

        Dim ParentValuesPerLine As Integer = bmdParent.Stride \ 4
        Dim ChildValuesPerLine As Integer = bmdChild.Stride \ 4

        Dim ParentData((bmdParent.Stride \ 4) * bmdParent.Height - 1) As Integer
        Dim ChildData((bmdChild.Stride \ 4) * bmdChild.Height - 1) As Integer

        Marshal.Copy(bmdParent.Scan0, ParentData, 0, ParentData.Length)
        Marshal.Copy(bmdChild.Scan0, ChildData, 0, ChildData.Length)

        If bmdParent IsNot Nothing Then Parent.UnlockBits(bmdParent)
        bmdParent = Nothing
        If bmdChild IsNot Nothing Then Child.UnlockBits(bmdChild)
        bmdChild = Nothing

        ' Create KMP-Table:
        Dim T(Child.Height - 1)() As Integer
        For i = 0 To Child.Height - 1
            T(i) = KMP_Table(ChildData, i * ChildValuesPerLine, ChildValuesPerLine)
        Next

        Dim line_c As Integer = 0
        Dim line_p As Integer = 0
        Dim found As Boolean

        While line_p <= Parent.Height - Child.Height
            line_c = 0
            Dim childoffset As Integer = line_c * ChildValuesPerLine
            Dim parentoffset As Integer = line_p * ParentValuesPerLine


            Dim m As Integer = -1
            While True
                m = KMP_Search(ParentData, parentoffset, ParentValuesPerLine, m + 1, ChildData, 0, ChildValuesPerLine, T(0))
                If m > -1 Then
                    ' first line found
                    Debug.Print("Possible match at {0},{1}", m, line_p)
                    found = True
                    Dim p = parentoffset + ParentValuesPerLine
                    Dim c = childoffset + ChildValuesPerLine
                    For i = 1 To Child.Height - 1
                        If KMP_Search(ParentData, p, ParentValuesPerLine, m, ChildData, childoffset, ChildValuesPerLine, T(i)) <> m Then
                            ' this line doesnt match
                            found = False
                            Exit For
                        End If
                        p += ParentValuesPerLine
                        c += ChildValuesPerLine
                    Next
                    If found Then
                        Debug.Print("Found match at {0},{1}", m, line_p)
                        Return New Point(m, line_p)
                    End If
                Else
                    Exit While
                End If
            End While

            line_p += 1
        End While

        'Catch ex As Exception
        'Throw
    Finally
        If bmdParent IsNot Nothing Then Parent.UnlockBits(bmdParent)
        If bmdChild IsNot Nothing Then Child.UnlockBits(bmdChild)
    End Try

End Function

Private Shared Function KMP_Search(ByVal S As Integer(), s0 As Integer, slen As Integer, m As Integer,
                                   ByVal W As Integer(), w0 As Integer, wlen As Integer,
                                   tbl() As Integer) As Integer

    Dim i As Integer = 0

    While m + i < slen
        If W(w0 + i) = S(s0 + m + i) Then
            If i = wlen - 1 Then Return m
            i += 1
        Else
            m = m + i - tbl(i)
            If tbl(i) > -1 Then
                i = tbl(i)
            Else
                i = 0
            End If
        End If

    End While

    Return -1


End Function

Private Shared Function KMP_Table(ByRef arr() As Integer, start As Integer, count As Integer) As Integer()

    Dim table(count - 1) As Integer
    table(0) = -1
    table(1) = 0

    Dim pos As Integer = 2
    Dim cnd As Integer = 0

    While pos < count - 1
        If arr(start + pos - 1) = arr(start + cnd) Then
            cnd += 1
            table(pos) = cnd
            pos += 1
        ElseIf cnd > 0 Then
            cnd = table(cnd)
        Else
            table(pos) = 0
            pos += 1
        End If
    End While

    Return table

End Function

End Class

我使用以下“基准”功能与 Abdias软件中的代码进行比较：

Private Sub Button1_Click_1(sender As Object, e As EventArgs) Handles Button1.Click

    Dim ofd As New OpenFileDialog
    If ofd.ShowDialog = Windows.Forms.DialogResult.OK Then
        Dim bm As Bitmap = Bitmap.FromStream(New MemoryStream(File.ReadAllBytes(ofd.FileName)))
        Dim bm2 As New Bitmap(bm.Width, bm.Height, PixelFormat.Format32bppArgb)
        Dim gr = Graphics.FromImage(bm2)
        gr.DrawImageUnscaled(bm, New Point(0, 0))
        Dim bm3 As New Bitmap(100, 100, PixelFormat.Format32bppArgb)
        gr = Graphics.FromImage(bm3)
        gr.DrawImage(bm2, New Rectangle(0, 0, 100, 100), New Rectangle(bm2.Width - 110, bm2.Height - 110, 100, 100), GraphicsUnit.Pixel)
        PictureBox1.Image = bm3
        Dim res As New List(Of Integer)
        For i = 1 To 10
            Dim stp = Stopwatch.StartNew
            Dim k = ImageFinder.Contains(bm2, bm3)
            stp.Stop()
            res.Add(stp.ElapsedMilliseconds)
        Next
        ListBox1.Items.Add(String.Format("KMP: Image = {3}x{4}, Min = {0}, Max = {1}, Avg = {2}", res.Min, res.Max, res.Average, bm2.Width, bm2.Height))
        res.Clear()
        For i = 1 To 10
            Dim stp = Stopwatch.StartNew
            Dim k = bm2.ContainsSO(bm3)
            stp.Stop()
            res.Add(stp.ElapsedMilliseconds)
        Next
        ListBox1.Items.Add(String.Format("SO: Image = {3}x{4}, Min = {0}, Max = {1}, Avg = {2}", res.Min, res.Max, res.Average, bm2.Width, bm2.Height))
    End If

End Sub

我测试了大型（8MP）和小型（1MP）照片（没有图纸，图标等），发现kmp版本大约比其他方法快2倍。对于在i7-2600上测试的8MP图像，绝对数字为40ms对75ms。结果可能取决于图像的类型，因为KMP（和其他人）赢了，当他们可以跳过更大的区域。