我正在修改Android Framework example以将MediaCodec生成的基本AAC流打包成独立的.mp4文件。我正在使用一个MediaMuxer个实例，其中包含MediaCodec个实例生成的一个AAC曲目。
但是，在调用mMediaMuxer.writeSampleData(trackIndex, encodedData, bufferInfo)时，我总是会收到错误消息：
E/MPEG4Writer﹕timestampUs 0 < lastTimestampUs XXXXX for Audio track
当我在mCodec.queueInputBuffer(...)中对原始输入数据进行排队时，我提供0作为每个框架示例的时间戳值（我也尝试使用具有相同结果的单调增加的时间戳值。我已成功编码原始相机使用相同的方法将帧转换为h264 / mp4文件。）
最相关的摘录：
private static void testEncoder(String componentName, MediaFormat format, Context c) {
int trackIndex = 0;
boolean mMuxerStarted = false;
File f = FileUtils.createTempFileInRootAppStorage(c, "aac_test_" + new Date().getTime() + ".mp4");
MediaCodec codec = MediaCodec.createByCodecName(componentName);
try {
codec.configure(
format,
null /* surface */,
null /* crypto */,
MediaCodec.CONFIGURE_FLAG_ENCODE);
} catch (IllegalStateException e) {
Log.e(TAG, "codec '" + componentName + "' failed configuration.");
}
codec.start();
try {
mMediaMuxer = new MediaMuxer(f.getAbsolutePath(), MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
} catch (IOException ioe) {
throw new RuntimeException("MediaMuxer creation failed", ioe);
}
ByteBuffer[] codecInputBuffers = codec.getInputBuffers();
ByteBuffer[] codecOutputBuffers = codec.getOutputBuffers();
int numBytesSubmitted = 0;
boolean doneSubmittingInput = false;
int numBytesDequeued = 0;
while (true) {
int index;
if (!doneSubmittingInput) {
index = codec.dequeueInputBuffer(kTimeoutUs /* timeoutUs */);
if (index != MediaCodec.INFO_TRY_AGAIN_LATER) {
if (numBytesSubmitted >= kNumInputBytes) {
Log.i(TAG, "queueing EOS to inputBuffer");
codec.queueInputBuffer(
index,
0 /* offset */,
0 /* size */,
0 /* timeUs */,
MediaCodec.BUFFER_FLAG_END_OF_STREAM);
if (VERBOSE) {
Log.d(TAG, "queued input EOS.");
}
doneSubmittingInput = true;
} else {
int size = queueInputBuffer(
codec, codecInputBuffers, index);
numBytesSubmitted += size;
if (VERBOSE) {
Log.d(TAG, "queued " + size + " bytes of input data.");
}
}
}
}
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
index = codec.dequeueOutputBuffer(info, kTimeoutUs /* timeoutUs */);
if (index == MediaCodec.INFO_TRY_AGAIN_LATER) {
} else if (index == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
MediaFormat newFormat = codec.getOutputFormat();
trackIndex = mMediaMuxer.addTrack(newFormat);
mMediaMuxer.start();
mMuxerStarted = true;
} else if (index == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
codecOutputBuffers = codec.getOutputBuffers();
} else {
// Write to muxer
ByteBuffer encodedData = codecOutputBuffers[index];
if (encodedData == null) {
throw new RuntimeException("encoderOutputBuffer " + index +
" was null");
}
if ((info.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) {
// The codec config data was pulled out and fed to the muxer when we got
// the INFO_OUTPUT_FORMAT_CHANGED status. Ignore it.
if (VERBOSE) Log.d(TAG, "ignoring BUFFER_FLAG_CODEC_CONFIG");
info.size = 0;
}
if (info.size != 0) {
if (!mMuxerStarted) {
throw new RuntimeException("muxer hasn't started");
}
// adjust the ByteBuffer values to match BufferInfo (not needed?)
encodedData.position(info.offset);
encodedData.limit(info.offset + info.size);
mMediaMuxer.writeSampleData(trackIndex, encodedData, info);
if (VERBOSE) Log.d(TAG, "sent " + info.size + " audio bytes to muxer with pts " + info.presentationTimeUs);
}
codec.releaseOutputBuffer(index, false);
// End write to muxer
numBytesDequeued += info.size;
if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
if (VERBOSE) {
Log.d(TAG, "dequeued output EOS.");
}
break;
}
if (VERBOSE) {
Log.d(TAG, "dequeued " + info.size + " bytes of output data.");
}
}
}
if (VERBOSE) {
Log.d(TAG, "queued a total of " + numBytesSubmitted + "bytes, "
+ "dequeued " + numBytesDequeued + " bytes.");
}
int sampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE);
int channelCount = format.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
int inBitrate = sampleRate * channelCount * 16; // bit/sec
int outBitrate = format.getInteger(MediaFormat.KEY_BIT_RATE);
float desiredRatio = (float)outBitrate / (float)inBitrate;
float actualRatio = (float)numBytesDequeued / (float)numBytesSubmitted;
if (actualRatio < 0.9 * desiredRatio || actualRatio > 1.1 * desiredRatio) {
Log.w(TAG, "desiredRatio = " + desiredRatio
+ ", actualRatio = " + actualRatio);
}
codec.release();
mMediaMuxer.stop();
mMediaMuxer.release();
codec = null;
}
更新：我发现我认为根本症状位于MediaCodec内。：
我将presentationTimeUs=1000发送给queueInputBuffer(...)，但在致电info.presentationTimeUs= 33219后收到MediaCodec.dequeueOutputBuffer(info, timeoutUs)。 fadden留下了与此行为相关的有用评论。
答案 0 :(得分：6)
感谢fadden的帮助，我在Github上获得了概念验证audio encoder和video+audio encoder。总结：
将AudioRecord的样本发送到MediaCodec + MediaMuxer包装器。使用audioRecord.read(...)时的系统时间可以充当音频时间戳，前提是您经常轮询以避免填充AudioRecord的内部缓冲区（以避免在您调用读取的时间和AudioRecord记录样本的时间之间漂移）。太糟糕了，AudioRecord没有直接传达时间戳......
// Setup AudioRecord
while (isRecording) {
audioPresentationTimeNs = System.nanoTime();
audioRecord.read(dataBuffer, 0, samplesPerFrame);
hwEncoder.offerAudioEncoder(dataBuffer.clone(), audioPresentationTimeNs);
}
请注意，AudioRecord only guarantees support for 16 bit PCM samples虽然MediaCodec.queueInputBuffer的输入为byte[]。将byte[]传递给audioRecord.read(dataBuffer,...)将 truncate 将16位样本拆分为8位。
我发现以这种方式进行轮询仍然偶尔会产生timestampUs XXX < lastTimestampUs XXX for Audio track错误，因此我添加了一些逻辑来跟踪bufferInfo.presentationTimeUs报告的mediaCodec.dequeueOutputBuffer(bufferInfo, timeoutMs)，并在调用之前根据需要进行调整{ {1}}。
答案 1 :(得分：2)
上面的回答https://stackoverflow.com/a/18966374/6463821中的代码也提供了timestampUs XXX < lastTimestampUs XXX for Audio track错误，因为如果您更快地从AudioRecord的buffer读取，则生成的timstamp之间的持续时间将小于音频之间的实际持续时间样品
因此，我对此问题的解决方案是生成第一个timstamp，每个下一个样本按样本的持续时间（depends on bit-rate, audio format, channel config)）增加时间戳。
BUFFER_DURATION_US = 1_000_000 * (ARR_SIZE / AUDIO_CHANNELS) / SAMPLE_AUDIO_RATE_IN_HZ;
...
long firstPresentationTimeUs = System.nanoTime() / 1000;
...
audioRecord.read(shortBuffer, OFFSET, ARR_SIZE);
long presentationTimeUs = count++ * BUFFER_DURATION + firstPresentationTimeUs;
从AudioRecord读取应该在单独的线程中，并且所有读取缓冲区都应该添加到队列中，而不必等待编码或任何其他操作，以防止丢失音频样本。
worker =
new Thread() {
@Override
public void run() {
try {
AudioFrameReader reader =
new AudioFrameReader(audioRecord);
while (!isInterrupted()) {
Thread.sleep(10);
addToQueue(
reader
.read());
}
} catch (InterruptedException e) {
Log.w(TAG, "run: ", e);
}
}
};
答案 2 :(得分：0)
问题发生，因为您无序地收到缓冲区： 尝试添加以下测试：
if(lastAudioPresentationTime == -1) {
lastAudioPresentationTime = bufferInfo.presentationTimeUs;
}
else if (lastAudioPresentationTime < bufferInfo.presentationTimeUs) {
lastAudioPresentationTime = bufferInfo.presentationTimeUs;
}
if ((bufferInfo.size != 0) && (lastAudioPresentationTime <= bufferInfo.presentationTimeUs)) {
if (!mMuxerStarted) {
throw new RuntimeException("muxer hasn't started");
}
// adjust the ByteBuffer values to match BufferInfo (not needed?)
encodedData.position(bufferInfo.offset);
encodedData.limit(bufferInfo.offset + bufferInfo.size);
mMuxer.writeSampleData(trackIndex.index, encodedData, bufferInfo);
}
encoder.releaseOutputBuffer(encoderStatus, false);