我正在尝试在android上解码音频并获取原始数据以应用过滤器。
我正在使用MediaExtractor从文件中提取编码后的数据，这似乎可行。
然后，我尝试将MediaExtractor docs和MediaCodec on Synchronous Processing using Buffers中的代码混合在一起，以提取数据并将其解码为块。
因此，我首先将解码器配置为采用extractor.getTrackFormat(0);
MediaExtractor extractor = new MediaExtractor();
String path = "...";
extractor.setDataSource(path);
MediaFormat format = extractor.getTrackFormat(0);
mAudioKeyMine = format.getString(MediaFormat.KEY_MIME);
extractor.selectTrack(0);
MediaCodec decoder;
decoder = MediaCodec.createDecoderByType(mAudioKeyMine);
decoder.configure(format, null, null, 0);
然后尝试获取数据：
public void getData(MediaExtractor extractor)
{
int offset = 0;
ByteBuffer inputBuffer = ByteBuffer.allocate(2048);
MediaFormat outputFormat = decoder.getOutputFormat();
Log.v(TAG, "outputFormat: " + outputFormat.toString());
decoder.start();
int index = decoder.dequeueInputBuffer(1000);
boolean sawInputEOS = false;
int sample = 0;
while (sample >= 0)
{
int inputBufferId = decoder.dequeueInputBuffer(1000);
if (inputBufferId >= 0)
{
inputBuffer = decoder.getInputBuffer(index);
sample = extractor.readSampleData(inputBuffer, 0);
long presentationTimeUs = 0;
if (sample < 0)
{
sawInputEOS = true;
sample = 0;
}
else
{
int trackIndex = extractor.getSampleTrackIndex();
presentationTimeUs = extractor.getSampleTime();
Log.v(TAG, "trackIndex: " + trackIndex + ", presentationTimeUs: " + presentationTimeUs);
Log.v(TAG, "sample: " + sample + ", offset: " + offset);
Log.v(TAG, "inputBuffer: " + inputBuffer.toString());
}
decoder.queueInputBuffer(inputBufferId, 0, sample, presentationTimeUs, sawInputEOS ? MediaCodec.BUFFER_FLAG_END_OF_STREAM : 0);
if (!sawInputEOS)
{
extractor.advance();
}
}
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
int outputBufferId = decoder.dequeueOutputBuffer(info, 1000);
Log.v(TAG, "info: " + info.toString());
if (outputBufferId >= 0)
{
ByteBuffer outputBuffer = decoder.getOutputBuffer(outputBufferId);
MediaFormat bufferFormat = decoder.getOutputFormat(outputBufferId);
Log.v(TAG, "option A");
Log.v(TAG, "outputBufferId: " + outputBufferId);
if (outputBuffer != null)
{
Log.v(TAG, "outputBuffer: " + outputBuffer.toString());
}
else
{
Log.v(TAG, "outputBuffer: null");
}
Log.v(TAG, "bufferFormat: " + bufferFormat.toString());
if (outputBuffer != null)
{
int cont = 0;
while (outputBuffer.hasRemaining())
{
int pos = outputBuffer.position();
byte data = outputBuffer.get();
// do something with the data
if (cont < 10)
{
Log.v(TAG, "outputBuffer: " + pos + " -> " + data);
}
cont++;
}
}
else
{
Log.v(TAG, "outputBuffer: null");
}
decoder.releaseOutputBuffer(outputBufferId, 0);
}
else if (outputBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED)
{
Log.v(TAG, "option B");
outputFormat = decoder.getOutputFormat();
Log.v(TAG, "outputFormat: " + outputFormat.toString());
}
Log.v(TAG, "extractor.advance()");
offset += sample;
}
Log.v(TAG, "end of track");
extractor.release();
extractor = null;
decoder.stop();
decoder.release();
}
但是在IllegalStateException行出现了一个错误int outputBufferId = decoder.dequeueOutputBuffer(info, 1000);。
我搜索了错误以及如何正确解码m4a，但是api 21弃用了大多数解决方案，现在我陷入了这个错误。
因此，有一个针对api 26/28的音频解码的示例，还是请有人解释如何正确进行？
整个项目都托管在GitHub上。
答案 0 :(得分：0)
我使用回调在异步模式下解决了这个问题。
基本工作流程是：
首先，我们需要进行一些初始化，将其放入用于解码和复制文件的类的构造函数中：
// inizialize the mediaExtractor and set the source file
mediaExtractor = new MediaExtractor();
mediaExtractor.setDataSource(fileName);
// select the first audio track in the file and return it's format
mediaFormat = null;
int i;
int numTracks = mediaExtractor.getTrackCount();
for (i = 0; i < numTracks; i++)
{
mediaFormat = mediaExtractor.getTrackFormat(i);
if (mediaFormat.getString(MediaFormat.KEY_MIME).startsWith("audio/"))
{
mediaExtractor.selectTrack(i);
break;
}
}
// we get the parameter from the mediaFormat
channelCount = mediaFormat.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
sampleRate = mediaFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE);
duration = mediaFormat.getLong(MediaFormat.KEY_DURATION);
mimeType = mediaFormat.getString(MediaFormat.KEY_MIME);
// we can get the minimum buffer size from audioTrack passing the parameter of the audio
// to keep it safe it's good practice to create a buffer that is 8 times bigger
int minBuffSize = AudioTrack.getMinBufferSize(sampleRate,
AudioFormat.CHANNEL_OUT_STEREO,
AudioFormat.ENCODING_PCM_16BIT);
// to reproduce the data we need to initialize the audioTrack, by passing the audio parameter
// we use the MODE_STREAM so we can put more data dynamically with audioTrack.write()
audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,
sampleRate,
AudioFormat.CHANNEL_OUT_STEREO,
AudioFormat.ENCODING_PCM_16BIT,
minBuffSize * 8,
AudioTrack.MODE_STREAM);
对于开发人员指南，我用来初始化audioTrack的方法已被弃用，但它对我有效，而新方法无效，因此出于本示例的目的，我保留了这种类型的初始化。
在初始化阶段之后，我们需要创建解码器，为其设置回调，然后启动解码器和audioTrack。 mediaCodec的回调为：
所以我们需要：
我的代码是：
// we get the mediaCodec by creating it using the mime_type extracted form the track
MediaCodec decoder = MediaCodec.createDecoderByType(mimeType);
// to decode the file in asynchronous mode we set the callbacks
decoder.setCallback(new MediaCodec.Callback()
{
private boolean mOutputEOS = false;
private boolean mInputEOS = false;
@Override
public void onInputBufferAvailable (@NonNull MediaCodec codec,
int index)
{
// if i reached the EOS i either the input or the output file i just skip
if (mOutputEOS | mInputEOS) return;
// i must use the index to get the right ByteBuffer from the codec
ByteBuffer inputBuffer = codec.getInputBuffer(index);
// if the codec is null i just skip and wait for another buffer
if (inputBuffer == null) return;
long sampleTime = 0;
int result;
// with this method i fill the inputBuffer with the data read from the mediaExtractor
result = mediaExtractor.readSampleData(inputBuffer, 0);
// the return parameter of readSampleData is the number of byte read from the file
// and if it's -1 it means that i reached EOS
if (result >= 0)
{
// if i read some bytes i can pass the index of the buffer, the number of bytes
// that are in the buffer and the sampleTime to the codec, so that it can decode
// that data
sampleTime = mediaExtractor.getSampleTime();
codec.queueInputBuffer(index, 0, result, sampleTime, 0);
mediaExtractor.advance();
}
else
{
// if i reached EOS i need to tell the codec
codec.queueInputBuffer(index, 0, 0, -1, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
mInputEOS = true;
}
}
@Override
public void onOutputBufferAvailable (@NonNull MediaCodec codec,
int index,
@NonNull MediaCodec.BufferInfo info)
{
// i can get the outputBuffer from the codec using the relative index
ByteBuffer outputBuffer = codec.getOutputBuffer(index);
// if i got a non null buffer
if (outputBuffer != null)
{
outputBuffer.rewind();
outputBuffer.order(ByteOrder.LITTLE_ENDIAN);
// i just need to write the outputBuffer into the audioTrack passing the number of
// bytes it contain and using the WRITE_BLOCKING so that this call will block
// until it doesn't finish to write the data
int ret = audioTrack.write(outputBuffer,
outputBuffer.remaining(),
AudioTrack.WRITE_BLOCKING);
}
// if the flags in the MediaCodec.BufferInfo contains the BUFFER_FLAG_END_OF_STREAM
// it mean that i reached EOS so i set mOutputEOS to true, and to assure
// that it remain true even if this callback is called again i use the logical or
mOutputEOS |= ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0);
// i always need to release the buffer i use so the system can recycle them and use
// it again
codec.releaseOutputBuffer(index, false);
// if i reached the end of the output stream i need to stop and release the codec
// and the extractor
if (mOutputEOS)
{
codec.stop();
codec.release();
mediaExtractor.release();
audioTrack.release();
}
}
@Override
public void onError (@NonNull MediaCodec codec,
@NonNull MediaCodec.CodecException e)
{
Timber.e(e, "mediacodec collback onError: %s", e.getMessage());
}
@Override
public void onOutputFormatChanged (@NonNull MediaCodec codec,
@NonNull MediaFormat format)
{
Timber.d("onOutputFormatChanged: %s", format.toString());
}
});
// now we can configure the codec by passing the mediaFormat and start it
decoder.configure(mediaFormat, null, null, 0);
decoder.start();
// also we need to start the audioTrack.
audioTrack.play();