MediaCodec无法解码H264编码的视频数据

时间:2019-01-28 21:38:33

标签: android h.264 rtp mediacodec

我正在开发一个通过RTP接收H264编码数据的应用程序,但我很难让Android的MediaCodec输出任何内容。我正在按照https://stackoverflow.com/a/7668578/10788248

所述对RTP数据包进行解包

编码帧重新组合后,我将其馈入出队的输入缓冲区。

当我将输入缓冲区放入队列时,我没有收到任何错误,但是解码器的回调永远不会调用onOutputBufferAvailable方法。

我能够调用它的唯一方法是传递流结束标志,然后输出大小为0。

我的问题是,“我显然有什么错吗?”和“哪些潜在问题可能导致输出缓冲区永远不可用,但编解码器没有抛出错误?”

代码已更新

LinkedList<DatagramPacket> packets = new LinkedList<>();
Thread socketReader = new Thread(() -> {
        try {
            DatagramSocket socket  = new DatagramSocket(videoPort);
            socket.connect(InetAddress.getByName(remoteAddress),remotePort);
            byte[] b;
            while (true){
                b = new byte[1500];
                DatagramPacket p = new DatagramPacket(b,1500);
                socket.receive(p);
                //Log.d(TAG,"RTP: "+bytesToHex(p.getData()));
                packets.add(p);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    });
@SuppressLint({"NewApi", "LocalSuppress"}) Thread packetHandler = new Thread(() -> {
        try {
            MediaCodec decoder = MediaCodec.createDecoderByType(MIME_TYPE);
            MediaFormat decoderFormat = MediaFormat.createVideoFormat(MIME_TYPE, 1280, 720);
            LinkedList<ByteBuffer> decoderInputs = new LinkedList<>();
            LinkedList<Integer> decoderIndices = new LinkedList<>();

            decoder.setCallback(new MediaCodec.Callback() {
                @Override
                public void onInputBufferAvailable(@NonNull MediaCodec codec, int index) {
                    decoderInputs.add(codec.getInputBuffer(index));
                    decoderIndices.add(new Integer(index));
                }

                @Override
                public void onOutputBufferAvailable(@NonNull MediaCodec codec, int index, @NonNull MediaCodec.BufferInfo info) {
                    Log.d(TAG,"OUTPUT AVAILABLE!!!!");
                    Log.d(TAG, String.valueOf(info.size));
                }

                @Override
                public void onError(@NonNull MediaCodec codec, @NonNull MediaCodec.CodecException e) {
                    Log.e(TAG,e.toString());
                }
                @Override
                public void onOutputFormatChanged(@NonNull MediaCodec codec, @NonNull MediaFormat format) {
                    Log.d(TAG,"FORMAT CHANGED");
                }
            });
            decoder.configure(decoderFormat, null,null,0);
            decoder.start();
            RTPFrame frame = new RTPFrame();
            boolean sps = false;
            boolean pps = false;
            while (true){
                if(decoderIndices.peek()!=null){
                    DatagramPacket packet = packets.poll();
                    if(packet!=null){
                        byte[] data = packet.getData();
                        if(frame==null||(!frame.isInitialized())){
                            frame = new RTPFrame(data);
                        }
                        else{
                            if(frame.isComplete()){
                                Integer index = null;
                                ByteBuffer decoderInput = null;
                                try {
                                    decoderInput = decoderInputs.poll().put(frame.getFrame());
                                    index = decoderIndices.poll();
                                    int size = frame.getFrameSize();
                                    if (frame.SPS) {
                                        Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                        decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                                        sps = true;
                                        pps = false;
                                    } else if (frame.PPS) {
                                        if(sps){
                                            Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                            decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                                            pps = true;
                                        }
                                    } else if (!frame.badFrame) {
                                        if(sps&&pps){
                                            Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                            decoder.queueInputBuffer(index, 0, size, frame.getTime(), 0);
                                        }
                                    }
                                    else{
                                        throw new RuntimeException();
                                    }
                                }
                                catch(Exception e){
                                    e.printStackTrace();
                                    if(index!=null){
                                        decoderIndices.push(index);
                                    }
                                    decoderInput.clear();
                                    decoderInputs.push(decoderInput);
                                }
                                frame = new RTPFrame();
                            }
                            else{
                                frame.addNALUnit(data);
                            }
                        }
                        if(frame.isComplete()){
                            Integer index = null;
                            ByteBuffer decoderInput = null;
                            try {
                                decoderInput = decoderInputs.poll().put(frame.getFrame());
                                index = decoderIndices.poll();
                                int size = frame.getFrameSize();
                                if (frame.SPS) {
                                    Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                    decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                                    sps = true;
                                    pps = false;
                                } else if (frame.PPS) {
                                    if(sps){
                                        Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                        decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                                        pps = true;
                                    }
                                } else if (!frame.badFrame) {
                                    if(sps&&pps){
                                        Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                        decoder.queueInputBuffer(index, 0, size, frame.getTime(), 0);
                                    }
                                }
                                else{
                                    throw new Exception("bad frame");
                                }
                            }
                            catch(Exception e){
                                e.printStackTrace();
                                if(index!=null){
                                    decoderIndices.push(index);
                                }
                                decoderInput.clear();
                                decoderInputs.push(decoderInput);
                            }
                            frame = new RTPFrame();
                        }
                    }
                }
            }
        } catch (Exception e) {e.printStackTrace();}
    });

这是我收到的一帧中所有NAL单元的样本,以及如何对它们进行拆包。

  

RTP:80 63 00 2D 27 0E 64 30 66 B4 BA 42 3C 81 E0 00 80 6F ...

     

RTP:80 63 00 2E 27 0E 64 30 66 B4 BA 42 3C 01 F3 0F E0 3F ...

     

RTP:80 63 00 2F 27 0E 64 30 66 B4 BA 42 3C 01 37 9B FA BA ...

     

RTP:80 63 00 30 27 0E 64 30 66 B4 BA 42 3C 01 7F EA 75 7C ...

     

RTP:80 63 00 31 27 0E 64 30 66 B4 BA 42 3C 01 FA D8 A9 FF ...

     

RTP:80 63 00 32 27 0E 64 30 66 B4 BA 42 3C 01 1B C5 BC C0 ...

     

RTP:80 63 00 33 27 0E 64 30 66 B4 BA 42 3C 01 0F F4 9A DE ...

     

RTP:80 63 00 34 27 0E 64 30 66 B4 BA 42 3C 01 F4 35 CD 28 ...

     

RTP:80 63 00 35 27 0E 64 30 66 B4 BA 42 3C 01 9E 45 70 13 ...

     

RTP:80 E3 00 36 27 0E 64 30 66 B4 BA 42 3C 41 0F 18 0D 83 ...

     

D / RTPreader:以2985639104长度= 12611:00 00 00 01 21拆包   E0 00 80 6F F0 B4 24 CD 5F 45 80 79 6E 0C ...

以下是拆包前后的示例SPS和PPS

  

RTP:80 63 00 00 5F DF A1 70 4F 2F 8A 3E 27 42 00 1F 8D 68 05 00 5B A1 00 00 03 00 01 00 00 03 00 1E 0F 10 7A 80

     

以1608491376的长度拆包= 27:00 00 01 27 42 00 1F 8D 68 05 00 5B A1 00 00 03 00 01 00 00 03 00 1E 0F 10 7A 80

     

以1608491376长度= 7:00 00 01 28 CE 32 48拆包

     

RTP:80 63 00 01 5F DF A1 70 4F 2F 8A 3E 28 CE 32 48

1 个答案:

答案 0 :(得分:2)

我要感谢@ greeble31和@ChrisBe在解决问题方面所提供的帮助。问题确实出在我的RTPFrame类上。具体来说,方法getFrame()遍历包含NAL单元数据的ByteBuffer列表,并使用put(ByteBuffer)将它们添加到bb中。这将position的{​​{1}}递增到等于bb

limit

public ByteBuffer getFrame() { size = 4; int nalCount = nalUnits.size(); for (int i = 0;i<nalCount;i++){ size+=nalUnits.get(i).data.length; } ByteBuffer bb = ByteBuffer.allocate(size); bb.put(new byte[]{0,0,0,1}); for(NALUnit unit:nalUnits){ bb.put(unit.data); } return bb; } 更改为return bb;可以解决此问题。