我正在从FFServer实例接收包含由libvo_aacenc
(44100hz 128kbps 2ch)编码的aac音频块的rtp数据包。我正试图在Android中逐个解码它们,并在块解码后立即播放。
Client.java
Player player = new Player();
//RTSP listener
@Override
public void onRTSPPacketReceived(RTPpacket packet) {
byte [] aac_chunk = packet.getpayload();
player.playAAC(aac_chunk);
}
Player.java
private MediaCodec decoder;
private AudioTrack audioTrack;
private MediaExtractor extractor;
public Player(){
extractor = new MediaExtractor();
audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,
44100, AudioFormat.CHANNEL_OUT_STEREO,
AudioFormat.ENCODING_PCM_16BIT,
44100,
AudioTrack.MODE_STREAM);
MediaFormat format = new MediaFormat();
format.setString(MediaFormat.KEY_MIME, "audio/mp4a-latm");
format.setInteger(MediaFormat.KEY_BIT_RATE, 128 * 1024);
format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 2);
format.setInteger(MediaFormat.KEY_SAMPLE_RATE, 44100);
format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectHE);
try{
decoder = MediaCodec.createDecoderByType("audio/mp4a-latm");
decoder.configure(format, null, null, 0);
} catch (IOException e) {
e.printStackTrace();
}
decoder.start();
audioTrack.play();
}
//Decode and play one aac_chunk
public void playAAC(byte [] data){
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
ByteBuffer[] inputBuffers = decoder.getInputBuffers();
ByteBuffer[] outputBuffers = decoder.getOutputBuffers();
int inIndex = decoder.dequeueInputBuffer(-1);
if (inIndex >= 0) {
ByteBuffer buffer = inputBuffers[inIndex];
buffer.put(data, 0, data.length);
int sampleSize = extractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
decoder.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
} else {
long presentationTimeUs = extractor.getSampleTime();
decoder.queueInputBuffer(inIndex, 0, sampleSize, presentationTimeUs, 0);
}
}
int outIndex = decoder.dequeueOutputBuffer(info, TIMEOUT);
while(outIndex >= 0){
ByteBuffer outBuffer = outputBuffers[outIndex];
byte[] decoded_chunk = new byte[info.size];
outBuffer.get(decoded_chunk); // Read the buffer all at once
outBuffer.clear();
//!! Decoded decoded_chunk.length = 0 !!
System.out.println("DECODED CHUNK SIZE: "+decoded_chunk.length);
//Instant play of the decoded chunk
audioTrack.write(decoded_chunk, info.offset, info.offset + info.size);
decoder.releaseOutputBuffer(outIndex, false);
break;
}
decoder.flush();
}
一开始,MediaCodec正确启动。
MediaCodec: (0xa5040280) start
MediaCodec: (0xa5040280) input buffers allocated
MediaCodec: (0xa5040280) numBuffers (4)
MediaCodec: (0xa5040280) output buffers allocated
MediaCodec: (0xa5040280) numBuffers (4)
问题
我实际上听不到声音。 MediaCodec正在运行,但看起来它没有将任何内容解码为输出缓冲区,因为decoded_chunk.length = 0
和outBuffer.limit() = 0
。
问题
我应该异步填充MediaCodec输入缓冲区吗?不幸的是,在我发现的关于这个问题的例子中没有找到任何东西:即时解码和回放。
我遵循以下示例:
答案 0 :(得分:0)
我已经在异步模式和MediaCodec.Callback中使用MediaCodec解决了这个问题,如官方文档here中所述,仅适用于Android minSdkVersion 21。
基本上我已经为我收到的每个RTP音频块使用了一个队列,然后每当MediaCodec缓冲状态改变时我就会收到通知。它实际上更容易处理解码器流程。
decoder.setCallback(new MediaCodec.Callback() {
@Override
public void onInputBufferAvailable(@NonNull MediaCodec mediaCodec, int i) {
//One InputBuffer is available to decode
while (true) {
if(queue.size() > 0) {
byte[] data = queue.removeFirst();
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
ByteBuffer buffer = mediaCodec.getInputBuffer(i);
buffer.put(data, 0, data.length);
mediaCodec.queueInputBuffer(i, 0, data.length, 0, 0);
break;
}
}
}
@Override
public void onOutputBufferAvailable(@NonNull MediaCodec mediaCodec, int i, @NonNull MediaCodec.BufferInfo info) {
//DECODING PACKET ENDED
ByteBuffer outBuffer = mediaCodec.getOutputBuffer(i);
byte[] chunk = new byte[info.size];
outBuffer.get(chunk); // Read the buffer all at once
outBuffer.clear();
audioTrack.write(chunk, info.offset, info.offset + info.size); // AudioTrack write data
mediaCodec.releaseOutputBuffer(i, false);
}
@Override
public void onError(@NonNull MediaCodec mediaCodec, @NonNull MediaCodec.CodecException e) {}
@Override
public void onOutputFormatChanged(@NonNull MediaCodec mediaCodec, @NonNull MediaFormat mediaFormat) {}
});