Question

我正在尝试提取图层激活以将其作为要素保存在本地。我还是CNN的新手，所以我想展示我的所作所为，我想知道我所做的事情是否正确：

public static void main(String[] args) throws IOException {
    ComputationGraph vgg16transfer = getComputationGraph();

    for (File file : new File(ImageClassifier.class.getClassLoader().getResource("mydirectory").getFile()).listFiles()) {
        Map<String, INDArray> stringINDArrayMap = extractTwo(file, vgg16transfer);
        //Extract the features from the last fully connected layers
        saveCompressed(file,stringINDArrayMap.get("fc2"));
    }
}

/**
 * Retrieves the VGG16 computation graph
 * @return ComputationGraph from the pretrained VGG16
 * @throws IOException
 */
public static ComputationGraph getComputationGraph() throws IOException {
    ZooModel zooModel = new VGG16();
    return (ComputationGraph) zooModel.initPretrained(PretrainedType.IMAGENET);
}

/**
 * Compresses the input INDArray and writes it to file
 * @param imageFile the original image file
 * @param array INDArray to be saved (features)
 * @throws IOException
 */
private static void saveCompressed(File imageFile, INDArray array) throws IOException {
    INDArray compress = BasicNDArrayCompressor.getInstance().compress(array);
    Nd4j.write(compress,new DataOutputStream(new FileOutputStream(new File("features/" + imageFile.getName()+ "feat"))));
}

/**
 * Given an input image and a ComputationGraph it calls the feedForward method after rescaling the image.
 * @param imageFile the image whose features need to be extracted 
 * @param vgg16 the ComputationGraph to be used.
 * @return a map of activations for each layer
 * @throws IOException
 */
public static Map<String, INDArray> extractTwo(File imageFile, ComputationGraph vgg16) throws IOException {
    // Convert file to INDArray
    NativeImageLoader loader = new NativeImageLoader(224, 224, 3);
    INDArray image = loader.asMatrix(imageFile);

    // Mean subtraction pre-processing step for VGG
    DataNormalization scaler = new VGG16ImagePreProcessor();
    scaler.transform(image);

    //Call the feedForward method to get a map of activations for each layer
    return vgg16.feedForward(image, false);
}

所以基本上我调用feedForward方法并从fc2层获取激活。

我有几个问题：

1）我写的代码是否确实提取了可以保存和存储的功能以供进一步使用？

2）我如何在提取的功能上进行PCA / Whitening？

3）有没有什么方法可以按照建议将其编码为VLAD但是这样的论文：https://arxiv.org/pdf/1707.00058.pdf

4）然后，我想比较保存的功能，我使用简单的欧几里德距离进行比较，虽然结果不是最好的，但它似乎正在工作。我应该做某种预处理，还是直接比较保存的功能？

感谢。

Answer 1

对于提取的功能，Nd4j有一个可以使用的PCA： https://github.com/deeplearning4j/nd4j/blob/master/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/dimensionalityreduction/PCA.java

但这有点多余。我会考虑直接使用转移学习。您不必手动执行任何操作。有关更多示例，请参阅此处的文档： https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-spark-examples/dl4j-spark/src/main/java/org/deeplearning4j/transferlearning/vgg16/TransferLearning.md

转移学习api将为您提供使用预训练模型所需的内容，同时仅修改它们以使用不同的输出层。我们甚至在上面的例子中用VGG16覆盖了它。

从deeplearning4j层提取特征

1 个答案: