Question

我对Flutter还是陌生的，基本上，我遵循在线教程来训练带有Google AutoML API的自定义图像标签模型，然后将该模型下载为三个文件（dict.txt，manifest.json，model.tflite），现在我正在尝试将其与我的flutter应用程序集成。

这是我的代码，用于加载和运行TFlite模型：

Future loadModel() async {
    try{
      res = await Tflite.loadModel(
          model: "assets/models/model.tflite",
          labels: "assets/models/dict.txt",
      );
      print("loading tf model...");
      print(res);
    }on PlatformException{
      print ("Failed to load model");
    }
  }

Future recognizeImageBinary(File image) async {
    var imageBytes = await image.readAsBytesSync();
    var bytes = imageBytes.buffer.asUint8List();
    img.Image oriImage = img.decodeJpg(bytes);
    img.Image resizedImage = img.copyResize(oriImage, height: 112, width: 112);

    var recognitions = await Tflite.runModelOnBinary(
      binary: imageToByteListUint8(resizedImage, 112),
      numResults: 2,
      threshold: 0.4,
      asynch: true
    );
    setState(() {
      _recognitions = recognitions;
    });
  }

根据教程，AutoML自定义训练模型的类型为Uint8，因此我使用以下函数对其进行了转换：

Uint8List imageToByteListUint8(img.Image image, int inputSize) {
    var convertedBytes = Uint8List(4 * inputSize * inputSize * 3);
    var buffer = Uint8List.view(convertedBytes.buffer);
    int pixelIndex = 0;
    for (var i = 0; i < inputSize; i++) {
      for (var j = 0; j < inputSize; j++) {
        var pixel = image.getPixel(j, i);
        buffer[pixelIndex++] = img.getRed(pixel);
        buffer[pixelIndex++] = img.getGreen(pixel);
        buffer[pixelIndex++] = img.getBlue(pixel);
      }
    }
    return convertedBytes.buffer.asUint8List();
  }

我有这样的例外情况：

E/AndroidRuntime( 6372): FATAL EXCEPTION: AsyncTask #2
E/AndroidRuntime( 6372): Process: com.soton.gca_app, PID: 6372
E/AndroidRuntime( 6372): java.lang.RuntimeException: An error occurred while executing doInBackground()
E/AndroidRuntime( 6372):    at android.os.AsyncTask$3.done(AsyncTask.java:318)
E/AndroidRuntime( 6372):    at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:354)
E/AndroidRuntime( 6372):    at java.util.concurrent.FutureTask.setException(FutureTask.java:223)
E/AndroidRuntime( 6372):    at java.util.concurrent.FutureTask.run(FutureTask.java:242)
E/AndroidRuntime( 6372):    at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:243)
E/AndroidRuntime( 6372):    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
E/AndroidRuntime( 6372):    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
E/AndroidRuntime( 6372):    at java.lang.Thread.run(Thread.java:760)
E/AndroidRuntime( 6372): Caused by: java.lang.IllegalArgumentException: Cannot convert between a TensorFlowLite tensor with type UINT8 and a Java object of type [[F (which is compatible with the TensorFlowLite type FLOAT32).
E/AndroidRuntime( 6372):    at org.tensorflow.lite.Tensor.throwIfTypeIsIncompatible(Tensor.java:316)
E/AndroidRuntime( 6372):    at org.tensorflow.lite.Tensor.throwIfDataIsIncompatible(Tensor.java:304)
E/AndroidRuntime( 6372):    at org.tensorflow.lite.Tensor.copyTo(Tensor.java:183)
E/AndroidRuntime( 6372):    at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:166)
E/AndroidRuntime( 6372):    at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:311)
E/AndroidRuntime( 6372):    at org.tensorflow.lite.Interpreter.run(Interpreter.java:272)
E/AndroidRuntime( 6372):    at sq.flutter.tflite.TflitePlugin$RunModelOnBinary.runTflite(TflitePlugin.java:478)
E/AndroidRuntime( 6372):    at sq.flutter.tflite.TflitePlugin$TfliteTask.doInBackground(TflitePlugin.java:419)
E/AndroidRuntime( 6372):    at sq.flutter.tflite.TflitePlugin$TfliteTask.doInBackground(TflitePlugin.java:393)
E/AndroidRuntime( 6372):    at android.os.AsyncTask$2.call(AsyncTask.java:304)
E/AndroidRuntime( 6372):    at java.util.concurrent.FutureTask.run(FutureTask.java:237)
E/AndroidRuntime( 6372):    ... 4 more

我现在真的很困惑，有人可以在这里帮忙吗？

Answer 1

@Shubham看来，即使我使用以下方法，异常也仍然存在：

Uint8List imageToByteListFloat32(img.Image image, int inputSize, double mean, double std) {
    var convertedBytes = Float32List(1 * inputSize * inputSize * 3 );
    var buffer = Float32List.view(convertedBytes.buffer);
    int pixelIndex = 0;
    for (var i = 0; i < inputSize; i++) {
      for (var j = 0; j < inputSize; j++) {
        var pixel = image.getPixel(j, i);
        buffer[pixelIndex++] = ((img.getRed(pixel) - mean) / std).toDouble();
        buffer[pixelIndex++] = ((img.getGreen(pixel) - mean) / std).toDouble();
        buffer[pixelIndex++] = ((img.getBlue(pixel) - mean) / std).toDouble();
      }
    }
    return convertedBytes.buffer.asUint8List();
  }

将自定义AutoML tflite模型与flutter应用程序集成

1 个答案: