我正在尝试使我的第一个图像分类模型正常工作,
VNClassificationObservation
无法正常工作
VNCoreMLFeatureValueObservation
是。
以下是有关我的模型的一些信息:
MLModelDescription: MLModelDescription inputDescriptionsByName: {
"input_1__0" = "input_1__0 : Image (Color, 299 x 299)";
} outputDescriptionsByName: {
"output_node0__0" = "output_node0__0 : MultiArray (MLMultiArrayDataTypeDouble, 43)";
} predictedFeatureName: (null)
根据文档:
VNClassificationObservation
This type of observation results from performing a VNCoreMLRequest image
analysis with a Core ML model whose role is classification (rather than
prediction or image-to-image processing).
Vision infers that an MLModel object is a classifier model if that model
predicts a single feature.
That is, the model's modelDescription object has a non-nil value for its
predictedFeatureName property.
起初,我假设当文档说“预测”时,他们指的是具有值预测的回归类型模型。但是现在我认为他们指的是softmax预测概率?因此,VNClassificationObservation不会输出softmax预测概率。
现在
VNCoreMLFeatureValueObservation:
Overview
This type of observation results from performing a VNCoreMLRequest image analysis with a Core ML model whose role is prediction rather than classification or image-to-image processing.
Vision infers that an MLModel object is a predictor model if that model predicts multiple features. You can tell that a model predicts multiple features when its modelDescription object has a nil value for its predictedFeatureName property, or when it inserts its output in an outputDescriptionsByName dictionary.
我对措辞感到困惑。这是否意味着多输入多输出模型? 不是分类,而是预测,这也有点令人困惑,但我假设 由于我得到的输出,softmax概率很大。
当我运行下面的代码时,我得到:
let request = VNCoreMLRequest(model: model) { [weak self] request, error in
guard let results = request.results as? [VNCoreMLFeatureValueObservation],
let topResult = results.first else {
fatalError("unexpected result type from VNCoreMLRequest")
DispatchQueue.main.async { [weak self] in
print("topResult!", topResult)
//print(model.debugDescription.outputDescriptionsByName)
}
}
let handler = VNImageRequestHandler(ciImage: image)
DispatchQueue.global(qos: .userInteractive).async {
do {
try handler.perform([request])
} catch {print(error)}
我得到了一堆价值:
topResult! Optional(<VNCoreMLFeatureValueObservation:
0x1c003f0c0> C99BC0A0-7722-4DDC-8FB8-C0FEB1CEEFA5 1 "MultiArray : Double 43
vector
[ 0.02323521859943867,0.03784361109137535,0.0327669121325016,0.02373981475830078,0.01920632272958755,0.01511944644153118,0.0268220379948616,0.00990589614957571,0.006585873663425446,0.02727104164659977,0.02337176166474819,0.0177282840013504,0.01582957617938519,0.01962342299520969,0.0335112139582634,0.01197215262800455,0.04638960584998131,0.0546870082616806,0.008390620350837708,0.02519697323441505,0.01038128975778818,0.02463733218610287,0.05725555866956711,0.02852404117584229,0.01987413503229618,0.02478211745619774,0.01224409975111485,0.03397252038121223,0.02300941571593285,0.02020683139562607,0.03740271925926208,0.01999092660844326,0.03210178017616272,0.02830206602811813,0.01122485008090734,0.01071082800626755,0.02285266295075417,0.01730070635676384,0.009790488518774509,0.01149104069918394,0.03331543132662773,0.01211327593773603,0.0193191897124052]" (1.000000))
如果这些确实是softmax概率,我将如何获取最大值的索引?我似乎无法使用.count
或类似的数组方法。
我试图将其强制转换为数组,但是这两个都不起作用 l
let values = topResult.featureValue as Array! (Can't convert...coercion)
let values = topResult as Array!
如果这些不是softmax值/概率,那么我会去获取 概率。价值观。我正在尝试获取前3个softmax概率的索引。
谢谢。
!!!更新!!!!!!!!:
在函数中尝试此操作: var localPrediction:字符串? 让topResult = results.first?.featureValue.multiArrayValue
DispatchQueue.main.async { () in
var max_value : Float32 = 0
for i in 0..<topResult!.count{
if max_value < topResult![i].floatValue{
max_value = topResult![i].floatValue
localPrediction = String(i)}
}
答案 0 :(得分:1)
如果模型是分类器,即mlmodel文件中的NeuralNetworkClassifier
,则输出为VNClassificationObservation
个对象。
如果您的模型不是分类器,即NeuralNetwork
或NeuralNetworkRegressor
,则输出是一个或多个VNCoreMLFeatureValueObservation
对象,其中包含最终层的输出。
因此,如果您希望在VNCoreMLFeatureValueObservation
中输出softmax,则需要确保模型的最后一层具有softmax。
要获取最大元素的索引和值,请使用:
func argmax(_ array: UnsafePointer<Double>, count: Int) -> (Int, Double) {
var maxValue: Double = 0
var maxIndex: vDSP_Length = 0
vDSP_maxviD(array, 1, &maxValue, &maxIndex, vDSP_Length(count))
return (Int(maxIndex), maxValue)
}
要使用此功能,请先将MLMultiArray的dataPointer
强制转换为UnsafePointer<Double>
,然后调用argmax()
函数:
let featurePointer = UnsafePointer<Double>(OpaquePointer(features.dataPointer))
let (maxIndex, maxValue) = argmax(featurePointer, 43)