根据形状和颜色比较两个图像

Question

我目前正在处理python脚本，在该脚本中我比较拍摄的图像（屏幕截图），然后将其裁剪到鼠标周围。然后将此图像与文件夹中图像的“库/数据库”进行比较，然后返回最相似的图像，并为该图像找到正确的数据，该图像我已经为其创建了Json数据库。

基本上，我正在尝试找出流行游戏“以撒的绑定”中特定项目的作用。我从屏幕上捕获图像，并说与图像库进行比较。然后在.json数据库中搜索该图像的名称，并找出其作用。我当前使用的方法是来自openCV的cv2.matchTemplate()，问题是该方法仅比较形状，而不比较颜色。在提到的游戏中，有很多相似的形状，最大的区别是颜色。所以我想知道应该使用什么方法/功能。

对不起，语法不好，英语不是我的母语。

裁剪图片的方法：

def crop_image(image_path):
sprite = cv2.imread(image_path)

sprite_resized = cv2.resize(sprite,(0,0),fx=10,fy=10) # resize for visualisation purposes

sprite_gray  = cv2.cvtColor(sprite_resized, cv2.COLOR_BGR2GRAY)

sprite_canny = cv2.Canny(sprite_gray, 60,30,3,3) # <- play around with these params.
sprite_canny = sprite_canny.astype(np.bool) # is now a masked array

# because sprite_canny is a masked array, we can change the pixels in the sprite_resized
# to a different color to display the result. here the canny edges are shown in red
sprite_resized[sprite_canny] = (0,0,255)

# clever part: get the ranges of the bounding x and y coordinates. for this, we can use
# .any(axis = 0 or 1) to check if there's a point along the x or y axis. read as: "any
# point along axis 0 or 1?"
mx = sprite_canny.any(axis = 0)
my = sprite_canny.any(axis = 1)

# with this, we can get the range of x and y coordinate and get the min and max from them.
# to calculate the range, we need the width and height in pixels, accessed using .shape
h, w = sprite_gray.shape
rx = np.arange(w)[mx]
ry = np.arange(h)[my]

x_min = np.min(rx)
y_min = np.min(ry)

x_max = np.max(rx)
y_max = np.max(ry)
#COMMENT OUT RETURN TOO SE DEFAULT SCRIPT
return cv2.resize(sprite_gray[y_min:y_max, x_min:x_max], IMAGE_SIZE)
# use these coordinates to display a green rectangle around the found sprite
cv2.rectangle(sprite_resized, (x_min,y_min), (x_max,y_max), (0,255,0), 2)

比较图像的方法：

def compare_images(image, library):
value = 0
max_value = 0
compared_image = str()

for file in os.listdir(library):
    template = f'{library}\\{file}'
    try:
        template = crop_image(template)
        value = cv2.matchTemplate(image, template, method=cv2.TM_CCOEFF)[0][0]
    except Exception: pass

    if value > max_value:
        max_value = value
        compared_image = file
return max_value, compared_image