我一直在尝试使用Simple Audio Recognition在iOS中实现Firebase's ML kit Tensorflow示例。我已经成功地训练了模型并将其转换为TFlite文件。该模型将Audio（wav）文件路径作为输入（[String]），并将预测作为输出（float32）。我的iOS代码非常简单
func initMLModel(){
/*Initializing local TFLite model*/
guard let modelPath = Bundle.main.path(forResource: "converted_model", ofType: "tflite") else {
return
}
let myLocalModel = LocalModelSource.init(modelName: "My", path: modelPath)
let registrationSuccessful = ModelManager.modelManager().register(myLocalModel)
let options = ModelOptions(cloudModelName: nil, localModelName: "My")
let interpreter = ModelInterpreter.modelInterpreter(options: options)
let ioOptions = ModelInputOutputOptions()
do {
try ioOptions.setInputFormat(index: 0, type: .unknown, dimensions: []) /*input is string path. Since string is not defined, setting it as unknown.*/
try ioOptions.setOutputFormat(index: 0, type: .float32, dimensions: [1,38]) /* output is 1 of 38 labelled classes*/
} catch let error as NSError {
print("Failed to set IO \(error.debugDescription)")
}
let inputs = ModelInputs()
var audioData = Data()
let audiopath = Bundle.main.path(forResource: "audio", ofType: "wav")
do {
audioData = try Data.init(contentsOf: URL.init(fileURLWithPath: audiopath!))
//try inputs.addInput(audioData) /*If the input type is direct audio data*/
try inputs.addInput([audiopath])
} catch let error as NSError {
print("Cannot get audio file data \(error.debugDescription)")
return
}
interpreter.run(inputs: inputs, options: ioOptions) { (outputs, error) in
if error != nil {
print("Error running the model \(error.debugDescription)")
return
}
do {
let output = try outputs!.output(index: 0) as? [[NSNumber]]
let probabilities = output?[0]
guard let labelsPath = Bundle.main.path(forResource: "conv_labels", ofType: "txt") else { return }
let fileContents = try? String.init(contentsOf: URL.init(fileURLWithPath: labelsPath))
guard let labels = fileContents?.components(separatedBy: "\n") else {return}
for i in 0 ..< labels.count {
if let probability = probabilities?[i] {
print("\(labels[i]) : \(probability)")
}
}
}catch let error as NSError {
print("Error in parsing the Output \(error.debugDescription)")
return
}
}
}
但是当我运行它时，我得到以下错误输出Failed to create a TFLite interpreter for the given model。示例应用程序的完整日志如下
2019-01-07 18:22:31.447917+0530 sample_core_ML[67500:3515789] - <AppMeasurement>[I-ACS036002] Analytics screen reporting is enabled. Call +[FIRAnalytics setScreenName:setScreenClass:] to set the screen name or override the default screen class name. To disable screen reporting, set the flag FirebaseScreenReportingEnabled to NO (boolean) in the Info.plist
2019-01-07 18:22:33.354449+0530 sample_core_ML[67500:3515686] libMobileGestalt MobileGestalt.c:890: MGIsDeviceOneOfType is not supported on this platform.
2019-01-07 18:22:34.789665+0530 sample_core_ML[67500:3515812] 5.15.0 - [Firebase/Analytics][I-ACS023007] Analytics v.50400000 started
2019-01-07 18:22:34.790814+0530 sample_core_ML[67500:3515812] 5.15.0 - [Firebase/Analytics][I-ACS023008] To enable debug logging set the following application argument: -FIRAnalyticsDebugEnabled (see )
2019-01-07 18:22:35.542993+0530 sample_core_ML[67500:3515823] [BoringSSL] nw_protocol_boringssl_get_output_frames(1301) [C1.1:2][0x7f9db0701d70] get output frames failed, state 8196
2019-01-07 18:22:35.543205+0530 sample_core_ML[67500:3515823] [BoringSSL] nw_protocol_boringssl_get_output_frames(1301) [C1.1:2][0x7f9db0701d70] get output frames failed, state 8196
2019-01-07 18:22:35.543923+0530 sample_core_ML[67500:3515823] TIC Read Status [1:0x0]: 1:57
2019-01-07 18:22:35.544070+0530 sample_core_ML[67500:3515823] TIC Read Status [1:0x0]: 1:57
2019-01-07 18:22:39.981492+0530 sample_core_ML[67500:3515823] 5.15.0 - [Firebase/MLKit][I-MLK002000] ModelInterpreterErrorReporter: Didn't find custom op for name 'DecodeWav' with version 1
2019-01-07 18:22:39.981686+0530 sample_core_ML[67500:3515823] 5.15.0 - [Firebase/MLKit][I-MLK002000] ModelInterpreterErrorReporter: Registration failed.
Failed to set IO Error Domain=com.firebase.ml Code=3 "input format 0 has invalid nil or empty dimensions." UserInfo={NSLocalizedDescription=input format 0 has invalid nil or empty dimensions.}
2019-01-07 18:22:40.604961+0530 sample_core_ML[67500:3515812] 5.15.0 - [Firebase/MLKit][I-MLK002000] ModelInterpreterErrorReporter: Didn't find custom op for name 'DecodeWav' with version 1
2019-01-07 18:22:40.605199+0530 sample_core_ML[67500:3515812] 5.15.0 - [Firebase/MLKit][I-MLK002000] ModelInterpreterErrorReporter: Registration failed.
Error running the model Optional(Error Domain=com.firebase.ml Code=2 "Failed to create a TFLite interpreter for the given model (/Users/minimaci73/Library/Developer/CoreSimulator/Devices/7FE413C1-3820-496A-B0CE-033BE2F3212A/data/Containers/Bundle/Application/868CB2FE-77D8-4B1F-8853-C2E17ECA63F2/sample_core_ML.app/converted_model.tflite)." UserInfo={NSLocalizedDescription=Failed to create a TFLite interpreter for the given model (/Users/minimaci73/Library/Developer/CoreSimulator/Devices/7FE413C1-3820-496A-B0CE-033BE2F3212A/data/Containers/Bundle/Application/868CB2FE-77D8-4B1F-8853-C2E17ECA63F2/sample_core_ML.app/converted_model.tflite).})
查看此行Didn't find custom op for name 'DecodeWav'时，我查看了受自定义支持的操作，发现Tensorflow默认已在audio_ops.cc中支持此操作。
详细信息
我的Tensorflow版本：1.12.0
环境：Conda
OS版本：Mac OSX Mojave 10.14.2
部署目标：ios 12.0
安装类型：Pod安装（pod'Firebase / MLModelInterpreter'）
但是我首先在v1.9.0中运行了我的训练模型。然后将Tensorflow更新到最新的v1.12.0，以运行TFLite转换器。两者都是主分支。
我的TFLite转换程序代码Python
import tensorflow as tf
graph_def_file = "my_frozen_graph.pb"
input_arrays = ["wav_data"]
output_arrays = ["labels_softmax"]
input_shape = {"wav_data" : [1,99,40,1]}
converter = tf.contrib.lite.TFLiteConverter.from_frozen_graph(
graph_def_file, input_arrays, output_arrays, input_shape)
converter.allow_custom_ops = True
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
答案 0 :(得分：0)
我在Firebase快速入门iOS存储库中发布了相同的问题，并且得到了以下回复DecodeWav op is never supported by TensorFlowLite。因此，尽管Tensorflow本身支持音频处理，但目前Tensorflow Lite不支持音频处理。