Question

我正在使用SurfaceTexture在后台运行设备的摄像头并从中获取帧。我看到onFrameAvailable回调不带框架，而是带SurfaceTexture。我想得到它的框架或图像表示，但我不知道该怎么做。当我搜索时，我发现我需要使用GraphicBuffer，但它似乎太复杂了，我不清楚如何使用它。

我也在这里看过解决方案：

Texture Image processing on the GPU?

Android SDK: Get raw preview camera image without displaying it

但目前尚不清楚如何在代码中执行此操作。这是我的代码：

public class BackgroundService extends Service  {

    private Camera camera = null;
    private int  NOTIFICATION_ID= 1;
    private static final String TAG = "OCVSample::Activity";
    // Binder given to clients
    private final IBinder mBinder = new LocalBinder();
    private WindowManager windowManager;
    private SurfaceTexture mSurfaceTexture= new SurfaceTexture (10);
    Intent intializerIntent;





    public class LocalBinder extends Binder {
        BackgroundService getService() {
            // Return this instance of this service so clients can call public methods
            return BackgroundService.this;
        }
    }//end inner class that returns an instance of the service.

    @Override
    public IBinder onBind(Intent intent) {
        intializerIntent = intent;
        return mBinder;
    }//end onBind.


    @Override
    public void onCreate() {

        Log.i(TAG, "onCreate is called");
        // Start foreground service to avoid unexpected kill

        startForeground(NOTIFICATION_ID, buildNotification());

        Thread thread = new Thread() {
            public void run() {

                camera = Camera.open(1);

                mSurfaceTexture.setOnFrameAvailableListener(new SurfaceTexture.OnFrameAvailableListener() {
                    @Override
                    public void onFrameAvailable(SurfaceTexture surfaceTexture) {
                        Log.i(TAG, "frame captured from texture");


                        if (camera!=null) {
                            //camera.setPreviewCallbackWithBuffer(null);
                            //camera.setPreviewCallback(null);
                            //camera.setOneShotPreviewCallback(null);

                            //if the following two lines are not called, many frames will be droped. 
                            camera.stopPreview();
                            camera.startPreview();
                        }


                    }
                });
                //now try to set the preview texture of the camera which is actually the  surfaceTexture that has just been created.

                try {
                    camera.setPreviewTexture(mSurfaceTexture);
                } catch (IOException e) {
                    Log.e(TAG, "Error in setting the camera surface texture");
                }



                camera.startPreview();
            }


        };
        thread.start();

    }

    private Notification buildNotification () {

        NotificationCompat.Builder notificationBuilder=new NotificationCompat.Builder(this);

        notificationBuilder.setOngoing(true); //this notification should be ongoing

        notificationBuilder.setContentTitle(getString(R.string.notification_title))
                .setContentText(getString (R.string.notification_text_and_ticker))
                .setSmallIcon(R.drawable.vecsat_logo)
                .setTicker(getString(R.string.notification_text_and_ticker));
        return(notificationBuilder.build());

    }


    @Override
    public void onDestroy() {

        Log.i(TAG, "surfaceDestroyed method");

        camera.stopPreview();
        //camera.lock();

        camera.release();
        mSurfaceTexture.detachFromGLContext();
        mSurfaceTexture.release();
        stopService(intializerIntent) ;
        //windowManager.removeView(surfaceView);

    }
}

如何在GPU或CPU上获取帧并处理它们？如果有一种方法可以在GPU上进行，那么从那里获得结果，这将是很好的，因为它似乎更有效。

谢谢。

Answer 1

SurfaceTexture将发送到Surface的任何内容转换为OpenGL ES“外部”纹理。如果要从软件访问这些像素，则需要将纹理渲染到帧缓冲区，然后使用glReadPixels()读取像素。其中一个例子是bigflake ExtractMpegFramesTest，它将解码视频的帧转换为PNG。

通过在GPU上进行所有处理可以实现更好的性能（参见例如this demo），但这并不总是可行的。

从SurfaceTexture（Android）获取帧

1 个答案: