TensorFlow:如何在创建批次时为图像数据集指定标签?

时间:2017-07-11 15:08:18

标签: python arrays image numpy tensorflow

是的,我搜索过SO,Reddit,GitHub,Google Plus等等。我在Windows 10 64位上使用TensorFlow运行Python 3。我的目标是阅读一堆图像并为其分配标签以进行培训。

我试图将我的标签列表转换为可用的"对象"为sess.run(train_step, feed_dict={imgs:batchX,lbls:batchY})。我的图像导入正常,因为在此之前我调用函数来创建批处理(下面的代码)。在函数中,我可以成功创建numpy数组图像。但是,我不知道从哪里开始分配我的标签。

我的labels.txt文件格式为

data/cats/cat (1) copy.png,1
data/cats/cat (2) copy.png,1
data/cats/cat (3) copy.png,1
and so on for about 300 lines

data/cats/cat (x) copy.png是文件,1是Class(在本例中是Cat)。该文件被读入一个名为labels_list的常规数组(或列表?),每行都是数组中的新元素。当我打印labels_list时,会显示

['data/cats/cat (1) copy.png,1' 'data/cats/cat (2) copy.png,1'
 'data/cats/cat (3) copy.png,1' 'data/cats/cat (4) copy.png,1'
 'data/cats/cat (5) copy.png,1' 'data/cats/cat (6) copy.png,1'
  (alot more lines of this)
 'data/cats/cat (295) copy.png,1' 'data/cats/cat (296) copy.png,1'
 'data/cats/cat (297) copy.png,1' 'data/cats/cat (298) copy.png,1']

我不知道如何为我的train_step制作一个可用的numpy数组(下面的代码)。我试过谷歌搜索,但大多数解决方案只使用带整数的标签列表,但我需要使用文件的路径。

任何帮助表示感谢,谢谢:)

代码:(和我的GitHub github.com/supamonkey2000/jm-uofa)

import tensorflow as tf
import numpy as np
import os
import sys
import cv2


content = [] # Where images are stored
labels_list = [] # Stores the image labels, still not 100% working


########## File opening function
with open("data/cats/files.txt") as ff:
    for line in ff:
        line = line.rstrip()
        content.append(line)
#################################

########## Labels opening function
with open("data/cats/labels.txt") as fff:
    for linee in fff:
        linee = linee.rstrip()
        labels_list.append(linee)
    labels_list = np.array(labels_list)
###############################


############ Function used to create batches for training
def create_batches(batch_size):
    images1 = [] # Array to hold images within the function
    for img1 in content: # Read the images from content[] in a loop
        thedata = cv2.imread(img1) # Load the image
        thedata = thedata.flatten() # Convert the image to a usable numpy array
        images1.append(thedata) # Append the image to the images1 array
    images1 = np.array(images1) # Convert images1[] to numpy array

    print(labels_list) # Debugging purposes

    while(True):
        for i in range(0,298,10):
            yield(images1[i:i+batch_size],labels_list[i:i+batch_size])
#########################################################


imgs = tf.placeholder(dtype=tf.float32,shape=[None,786432]) # Images placeholder
lbls = tf.placeholder(dtype=tf.float32,shape=[None,10]) # Labels placeholder

W = tf.Variable(tf.zeros([786432,10])) # Weights
b = tf.Variable(tf.zeros([10])) # Biases

y_ = tf.nn.softmax(tf.matmul(imgs,W) + b) # Something complicated

cross_entropy = tf.reduce_mean(-tf.reduce_sum(lbls * tf.log(y_),reduction_indices=[1])) # Cool spacey sounding thing that does cool stuff
train_step = tf.train.GradientDescentOptimizer(0.05).minimize(cross_entropy) # When this is called use the GDO to train the model

sess = tf.InteractiveSession() # Setup the session
tf.global_variables_initializer().run() # Initialize the variables

############################## Training steps for teaching the model
for i in range(10000): # Run for 10,000 steps
    for (batchX,batchY) in create_batches(10): # Call a batch to be used
        sess.run(train_step, feed_dict={imgs:batchX, lbls: batchY}) # Train the model with the batch (THIS IS GIVING ME TONS OF ISSUES)
###################################################################


correct_prediction = tf.equal(tf.argmax(y_,1),tf.argmax(lbls,1)) # Find out if the program tested properly (I think?)
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) # Find the accuracy of the model

print(sess.run(accuracy, feed_dict={imgs:content, lbls:labels_list})) # Print the accuracy of the model !!! imgs:content may be incorrect, must look into it

1 个答案:

答案 0 :(得分:1)

据我所知,你有2个档案 - data / cats / files.txt其中包含文件的URL - data / cats / labels.txt其中还包含文件的URL和相应的标签。

我建议只使用标签文件,因为网址和标签在这里链接。

现在继续贴标签。

当您阅读labels.txt文件时,已经创建了输出标签。我在代码中添加了一些注释。

import re
import numpy
import cv2

label_map = [] # the map that holds the link between the label and the one_hot_encoded label
file_info = [] # holds all your file locations and the label of the file
#read all the lines. A line should look like this: mycatimage.png,1 
with open('labels.txt') as f:
    rows = [re.split(",", line.rstrip("\n")) for line in f]

for row in rows:
    file_info.append(row)

label_column = [line[1] for line in rows] # line[1] is based on your 'data/cats/cat (1) copy.png,1' example where the label is the second index

unique_labels = list(set(label_column)) # set gives unique values
# now the onehot encoding of the labels which basically means everything 0 except 1
for label in unique_labels:
    output_values = np.zeros(len(unique_labels), dtype=np.int)
    output_values [unique_labels.index(label)] = 1
    label_map.append({'name': label , 'value': output_values })


# Write the found labels to the label file if you want for later use. We will use the label_map variable for now
with open("mylabelfile.txt", 'w+') as lf:
    for label in label_map:
        lf.write(label['name'] + ',' + ','.join(str(x) for x in label['value']) + "\n")  # writes --> Christina,0,0,1,0\n

现在进入批处理功能:)

def get_one_hot_encoded_array_for_label(label):
    for existing in label_map:
        if existing['name'] == label:
            return existing['value']

def create_batches(batch_size):
    images1 = [] # Array to hold images within the function
    labels_list = []
    for img1 in file_info: # Read the images from file_info[] in a loop
        image_location = img1[0]
        image_label = img1[1]
        thedata = cv2.imread(image_location ) # Load the image
        thedata = thedata.flatten() # Convert the image to a usable numpy array
        images1.append(thedata) # Append the image to the images1 array
        outputvalues = get_one_hot_encoded_array_for_label(image_label )
        labels_list.append(outputvalues) # where we fill the labels list with the one hot encoded values.
    images1 = np.array(images1) # Convert images1[] to numpy array
    labels_list = np.array(labels_list)
    print(labels_list) # Debugging purposes

    while(True):
        for i in range(0,298,10):
            yield(images1[i:i+batch_size],labels_list[i:i+batch_size])

这应该提供一个热门编码值batchY。基于我自己的网络写了这个,但没有用图像数据测试它。你能确认它是否有效或告诉它在哪里断裂?如果有什么不清楚请问:)