我正在开发一个通过RTP接收H264编码数据的应用程序，但我很难让Android的MediaCodec输出任何内容。我正在按照https://stackoverflow.com/a/7668578/10788248
所述对RTP数据包进行解包编码帧重新组合后，我将其馈入出队的输入缓冲区。
当我将输入缓冲区放入队列时，我没有收到任何错误，但是解码器的回调永远不会调用onOutputBufferAvailable方法。
我能够调用它的唯一方法是传递流结束标志，然后输出大小为0。
我的问题是，“我显然有什么错吗？”和“哪些潜在问题可能导致输出缓冲区永远不可用，但编解码器没有抛出错误？”
代码已更新
LinkedList<DatagramPacket> packets = new LinkedList<>();
Thread socketReader = new Thread(() -> {
try {
DatagramSocket socket = new DatagramSocket(videoPort);
socket.connect(InetAddress.getByName(remoteAddress),remotePort);
byte[] b;
while (true){
b = new byte[1500];
DatagramPacket p = new DatagramPacket(b,1500);
socket.receive(p);
//Log.d(TAG,"RTP: "+bytesToHex(p.getData()));
packets.add(p);
}
} catch (Exception e) {
e.printStackTrace();
}
});
@SuppressLint({"NewApi", "LocalSuppress"}) Thread packetHandler = new Thread(() -> {
try {
MediaCodec decoder = MediaCodec.createDecoderByType(MIME_TYPE);
MediaFormat decoderFormat = MediaFormat.createVideoFormat(MIME_TYPE, 1280, 720);
LinkedList<ByteBuffer> decoderInputs = new LinkedList<>();
LinkedList<Integer> decoderIndices = new LinkedList<>();
decoder.setCallback(new MediaCodec.Callback() {
@Override
public void onInputBufferAvailable(@NonNull MediaCodec codec, int index) {
decoderInputs.add(codec.getInputBuffer(index));
decoderIndices.add(new Integer(index));
}
@Override
public void onOutputBufferAvailable(@NonNull MediaCodec codec, int index, @NonNull MediaCodec.BufferInfo info) {
Log.d(TAG,"OUTPUT AVAILABLE!!!!");
Log.d(TAG, String.valueOf(info.size));
}
@Override
public void onError(@NonNull MediaCodec codec, @NonNull MediaCodec.CodecException e) {
Log.e(TAG,e.toString());
}
@Override
public void onOutputFormatChanged(@NonNull MediaCodec codec, @NonNull MediaFormat format) {
Log.d(TAG,"FORMAT CHANGED");
}
});
decoder.configure(decoderFormat, null,null,0);
decoder.start();
RTPFrame frame = new RTPFrame();
boolean sps = false;
boolean pps = false;
while (true){
if(decoderIndices.peek()!=null){
DatagramPacket packet = packets.poll();
if(packet!=null){
byte[] data = packet.getData();
if(frame==null||(!frame.isInitialized())){
frame = new RTPFrame(data);
}
else{
if(frame.isComplete()){
Integer index = null;
ByteBuffer decoderInput = null;
try {
decoderInput = decoderInputs.poll().put(frame.getFrame());
index = decoderIndices.poll();
int size = frame.getFrameSize();
if (frame.SPS) {
Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
sps = true;
pps = false;
} else if (frame.PPS) {
if(sps){
Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
pps = true;
}
} else if (!frame.badFrame) {
if(sps&&pps){
Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
decoder.queueInputBuffer(index, 0, size, frame.getTime(), 0);
}
}
else{
throw new RuntimeException();
}
}
catch(Exception e){
e.printStackTrace();
if(index!=null){
decoderIndices.push(index);
}
decoderInput.clear();
decoderInputs.push(decoderInput);
}
frame = new RTPFrame();
}
else{
frame.addNALUnit(data);
}
}
if(frame.isComplete()){
Integer index = null;
ByteBuffer decoderInput = null;
try {
decoderInput = decoderInputs.poll().put(frame.getFrame());
index = decoderIndices.poll();
int size = frame.getFrameSize();
if (frame.SPS) {
Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
sps = true;
pps = false;
} else if (frame.PPS) {
if(sps){
Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
pps = true;
}
} else if (!frame.badFrame) {
if(sps&&pps){
Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
decoder.queueInputBuffer(index, 0, size, frame.getTime(), 0);
}
}
else{
throw new Exception("bad frame");
}
}
catch(Exception e){
e.printStackTrace();
if(index!=null){
decoderIndices.push(index);
}
decoderInput.clear();
decoderInputs.push(decoderInput);
}
frame = new RTPFrame();
}
}
}
}
} catch (Exception e) {e.printStackTrace();}
});
这是我收到的一帧中所有NAL单元的样本，以及如何对它们进行拆包。
RTP：80 63 00 2D 27 0E 64 30 66 B4 BA 42 3C 81 E0 00 80 6F ...
RTP：80 63 00 2E 27 0E 64 30 66 B4 BA 42 3C 01 F3 0F E0 3F ...
RTP：80 63 00 2F 27 0E 64 30 66 B4 BA 42 3C 01 37 9B FA BA ...
RTP：80 63 00 30 27 0E 64 30 66 B4 BA 42 3C 01 7F EA 75 7C ...
RTP：80 63 00 31 27 0E 64 30 66 B4 BA 42 3C 01 FA D8 A9 FF ...
RTP：80 63 00 32 27 0E 64 30 66 B4 BA 42 3C 01 1B C5 BC C0 ...
RTP：80 63 00 33 27 0E 64 30 66 B4 BA 42 3C 01 0F F4 9A DE ...
RTP：80 63 00 34 27 0E 64 30 66 B4 BA 42 3C 01 F4 35 CD 28 ...
RTP：80 63 00 35 27 0E 64 30 66 B4 BA 42 3C 01 9E 45 70 13 ...
RTP：80 E3 00 36 27 0E 64 30 66 B4 BA 42 3C 41 0F 18 0D 83 ...
D / RTPreader：以2985639104长度= 12611：00 00 00 01 21拆包 E0 00 80 6F F0 B4 24 CD 5F 45 80 79 6E 0C ...
以下是拆包前后的示例SPS和PPS
RTP：80 63 00 00 5F DF A1 70 4F 2F 8A 3E 27 42 00 1F 8D 68 05 00 5B A1 00 00 03 00 01 00 00 03 00 1E 0F 10 7A 80
以1608491376的长度拆包= 27：00 00 01 27 42 00 1F 8D 68 05 00 5B A1 00 00 03 00 01 00 00 03 00 1E 0F 10 7A 80
以1608491376长度= 7：00 00 01 28 CE 32 48拆包
RTP：80 63 00 01 5F DF A1 70 4F 2F 8A 3E 28 CE 32 48
答案 0 :(得分：2)
我要感谢@ greeble31和@ChrisBe在解决问题方面所提供的帮助。问题确实出在我的RTPFrame类上。具体来说，方法getFrame()遍历包含NAL单元数据的ByteBuffer列表，并使用put(ByteBuffer)将它们添加到bb中。这将position的{{1}}递增到等于bb。
limit将public ByteBuffer getFrame() {
size = 4;
int nalCount = nalUnits.size();
for (int i = 0;i<nalCount;i++){
size+=nalUnits.get(i).data.length;
}
ByteBuffer bb = ByteBuffer.allocate(size);
bb.put(new byte[]{0,0,0,1});
for(NALUnit unit:nalUnits){
bb.put(unit.data);
}
return bb;
}
更改为return bb;可以解决此问题。