使用CNN训练大量数据

时间:2018-12-21 12:07:01

标签: python tensorflow training-data tflearn

作为机器学习的新手,我尝试搜索许多用于训练分类器的方法以及如何使数据可读的方法。到目前为止,我所知道的是,对于分类器而言,标签是最重要的,这是显而易见的。我的问题是,我在其他文件夹中有超过300000张图像和标签的海量数据集,解释了每个图像的侧面和边界框,我还有其他数据的信息,这些信息不在其他文件夹中,而在其他文件夹中.mat文件和每个.mat文件均在图像中包含汽车的品牌或型号名称。因为到目前为止,我一直以numpy格式转换图像和标签数据,并将它们附加到training_data中,所以我应该对其他数据进行处理,以便也可以使用标签和图像数据对其进行训练。

您的答案将受到高度赞赏。

在下面,我将对文件夹进行更多说明,因此大家都对数据有更多的了解。如果你们可以的话,我只需要理论上的回答步骤即可。

数据说明

文件夹和文件的描述如下:

-image:
  Stores all full car images in the path format 'make_id/model_id/released_year/image_name.jpg'.
-label:
  Stores all labels to the full car images in the path format 'make_id/model_id/released_year/image_name.txt'. Each label file has three lines. The first line is a number which is the viewpoint annotation (-1 - uncertain, 1 - front, 2 - rear, 3 - side, 4 - front-side, 5 - rear-side). The second line is the number of the bounding boxes, which is all '1' in the current release. The third line is the coordinates of the bounding box in the format 'x1 y1 x2 y2' in pixels, where 1 <= x1 < x2 <= image_width, and 1 <= y1 < y2 <= image_height. 
-misc:
  -attributes.txt:
    Each line is the attribute annotation for one model which is in the format 'model_id maximum_speed displacement door_number seat_number type'. For car types, a number from 1~12 corresponds to a specific type, which is described in 'car_type.mat'. Unavailable attributes are denoted by '0' or '0.0'.
  -make_model_name.mat
    Cell array 'make_names' provides the projections from 'make_id' to make names, and cell array 'model_names' provides the projections from 'model_id' to model names.
-part:
  Stores all part images in the path format 'make_id/model_id/released_year/part_id/image_name.jpg'. The correspondance of 'part_id' and part names are: 1 - headlight, 2 - taillight, 3 - fog light, 4 - air intake, 5 - console, 6 - steering wheel, 7 - dashboard, and 8 - gear lever.
-train_test_split:
  This folder generally provides all the train/test subsets used in the paper.
  -classification
    Stores the train/test lists for the classification task with full car images in the paper.
  -part:
    Stores the train/test lists for the classification task with car part in the paper.
  -verification:
    'verification_train.txt' is the image list for training the verification models which is also for testing attribute prediction. 'verification_pairs_easy.txt', 'verification_pairs_medium.txt', and 'verification_pairs_hard.txt' are the three sets with different difficulties for testing car verification models. Each line of 'verification_pairs_XXX.txt' is in the format of 'path_to_image_1 path_to_image_2 label' where label is '1' for positive pairs and is '0' for negative pairs.

谢谢。

1 个答案:

答案 0 :(得分:1)

您可以做的是使其他数据数值化。

假设您有3种不同的汽车型号:FerrariTeslaLambo。您可以定义,如果汽车是Ferrari,则数字是0。如果为Tesla,则为1;如果为Lambo,则为2。现在,当您加载标签时,还加载杂项文件夹数据并按照上面的定义进行交换:Ferrari = 0, Tesla = 1, Lambo = 2现在,您可以将其附加到标签向量上,并指导网络预测0,{{ 1}}或1(用于汽车模型)。网络做出预测后,您可以断言预测是否为真(例如,如果NN预测了2,则意味着图像中的汽车是1)。

您可以将这种方法应用于misc文件夹中的任何其他非数字功能。称为Tesla-将非数字值转换为数字。