我设法获得以下代码,从目录中获取图像列表,使用ImageAI Python module对它们进行对象检测,然后将“处理后的”图像保存到新目录中。当我仅指定一个附加线程来运行它时,使用下面的代码可以很好地实现此目的。
当我尝试增加线程范围时,它开始引发此异常:
无法将feed_dict键解释为张量:张量 Tensor(“ Placeholder:0”,shape =(7,7,3,64),dtype = float32)不是 该图的元素。
我无法在docs中找到无法进行多线程处理的任何内容。
在有帮助的情况下,一些其他控制台输出(尝试5个线程时输出):
C:\Users\Me\Desktop\Project>cd c:\Users\Me\Desktop\Project && cmd /C "set "PYTHONIOENCODING=UTF-8" && set "PYTHONUNBUFFERED=1" && C:\Users\Me\AppData\Local\Programs\Python\Python36\python.exe c:\Users\Me\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\ptvsd_launcher.py --default --client --host localhost
--port 61076 c:\Users\Me\Desktop\Project\app.py "
Using TensorFlow backend.
2019-01-02 21:52:37.108429: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: GeForce GTX 1070 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.683
pciBusID: 0000:03:00.0
totalMemory: 8.00GiB freeMemory: 6.61GiB
2019-01-02 21:52:37.113537: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-01-02 21:52:38.396715: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-01-02 21:52:38.399994: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-01-02 21:52:38.402189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-01-02 21:52:38.404115: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6368
MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070 Ti, pci bus id: 0000:03:00.0, compute capability: 6.1)
2019-01-02 21:52:38.419623: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-01-02 21:52:38.422946: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-01-02 21:52:38.425552: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-01-02 21:52:38.427139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-01-02 21:52:38.428894: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6368
MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070 Ti, pci bus id: 0000:03:00.0, compute capability: 6.1)
2019-01-02 21:52:38.436241: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-01-02 21:52:38.439749: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-01-02 21:52:38.442604: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-01-02 21:52:38.444264: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-01-02 21:52:38.445908: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6368
MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070 Ti, pci bus id: 0000:03:00.0, compute capability: 6.1)
2019-01-02 21:52:38.460772: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-01-02 21:52:38.463329: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-01-02 21:52:38.466661: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-01-02 21:52:38.468411: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-01-02 21:52:38.470123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6368
MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070 Ti, pci bus id: 0000:03:00.0, compute capability: 6.1)
2019-01-02 21:52:38.476201: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-01-02 21:52:38.478613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-01-02 21:52:38.484133: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-01-02 21:52:38.493996: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-01-02 21:52:38.496042: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6368
MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070 Ti, pci bus id: 0000:03:00.0, compute capability: 6.1)
Backend TkAgg is interactive backend. Turning interactive mode on.
----Errors out on this line----
单线程的工作代码:
import os
import cv2
import threading
import subprocess
import tensorflow as tf
from queue import Queue
from imageai.Detection import ObjectDetection
cwd = os.getcwd()
image_dir = os.path.join(cwd, 'TrainingImages')
processed_dir = os.path.join(cwd, 'ProcessedImages')
def process_image(image_filename):
detector = ObjectDetection()
detector.setModelTypeAsRetinaNet()
detector.setModelPath(os.path.join(cwd, "resnet50_coco_best_v2.0.1.h5"))
detector.loadModel(detection_speed="fast")
if image_filename.endswith(".JPG"):
try:
detections = detector.detectObjectsFromImage(input_image=os.path.join(image_dir, image_filename), output_image_path=os.path.join(processed_dir, 'processed_' + image_filename))
return detections
except ValueError as e:
print("Error processing: {}: {}".format(os.path.join(image_dir, image_filename), e))
return False
def worker(q):
''' The worker thread pulls an item from the queue and processes it '''
while True:
image_filename = q.get()
detections = process_image(image_filename)
if detections:
for obj in detections:
print("{} : {}".format(obj["name"], obj["percentage_probability"]))
else:
print("No objects detected")
q.task_done()
if __name__ == '__main__':
# Get all files in image directory
image_list = os.listdir(image_dir)
# Create Queue for images
q = Queue()
for image_filename in image_list:
q.put(image_filename)
for i in range(1):
t = threading.Thread(target=worker, args=(q, ))
t.daemon = True
t.start()
q.join()
项目利用了以下内容:
Python version 3.6.5
Package Version
------------------- ---------
absl-py 0.6.1
astor 0.7.1
astroid 1.6.4
certifi 2018.4.16
chardet 3.0.4
colorama 0.3.9
cycler 0.10.0
gast 0.2.0
grpcio 1.17.1
h5py 2.9.0
idna 2.6
imageai 2.0.2
isort 4.3.4
Keras 2.2.4
Keras-Applications 1.0.6
Keras-Preprocessing 1.0.5
kiwisolver 1.0.1
lazy-object-proxy 1.3.1
Markdown 3.0.1
matplotlib 3.0.2
mccabe 0.6.1
numpy 1.15.4
opencv-python 3.4.5.20
Pillow 5.4.0
pip 18.1
pipenv 2018.5.18
protobuf 3.6.1
pylint 1.9.1
pyparsing 2.3.0
python-dateutil 2.7.5
PyYAML 3.13
requests 2.18.4
scipy 1.2.0
setuptools 39.0.1
six 1.11.0
tensorboard 1.12.1
tensorflow-gpu 1.12.0
termcolor 1.1.0
urllib3 1.22
virtualenv 16.0.0
virtualenv-clone 0.3.0
Werkzeug 0.14.1
wheel 0.32.3
wrapt 1.10.11
问题:
有人可以帮助我了解如何对上述代码进行多线程处理以允许异步处理多个图像吗?
我试图提供尽可能多的信息。
一些附加信息:它似乎仍然可以在多个线程中运行;但是,第一个之后的每个线程都会在上面抛出该错误。然后继续处理第一个图像,然后基本上“挂起”(什么都不做)。我还不太熟悉该术语,还不知道Tensor“图”的含义或要进行进一步故障排除的搜索内容。
这是5个线程的输出:
2019-01-02 22:23:53.417469: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6368 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070 Ti, pci bus id: 0000:03:00.0, compute capability: 6.1)
Error thrown: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder:0", shape=(7, 7, 3, 64), dtype=float32) is not an element of this graph.
Error thrown: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder:0", shape=(7, 7, 3, 64), dtype=float32) is not an element of this graph.
Error thrown: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder:0", shape=(7, 7, 3, 64), dtype=float32) is not an element of this graph.
Error thrown: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder:0", shape=(7, 7, 3, 64), dtype=float32) is not an element of this graph.
person : 78.54841947555542
truck : 56.28065466880798