最近我想玩一些音频内容并寻找一些我找到CSCore的音频库。因为我是在抽象级别编码声音的新手（对我来说很低），我正在努力解决播放方面的一些问题。
我的目标是能够同时在内存声音缓冲区中多次播放（并行）。如何在CSCore中完成？
起初，我一直在努力平行演奏（两种不同的声音）。我最终理解在GitHub上阅读CSCore样本及其来源是为了使用单个SoundOut并行播放任何内容我需要创建执行混音的SampleSource或WaveSource。因此，在GitHub中提供的混音器样本的帮助下，我已经实现了我自己的简单混音器（即混合多个ISampleSource-s的ISampleSource）。大！它从两个不同的来源同时播放两种声音。
但这是我被卡住的地方。将单个声音从磁盘加载到内存我希望能够多次播放，可能重叠（并行播放）。就像我有60秒的声音，我开始播放，5秒钟后，我想再开始一次，那些与你能听到的重叠。
当我刚刚在混音器列表中添加了两次相同的ISampleSource（我想要播放的声音）时，我得到了奇怪的输出（声音播放速度更快，出现故障）。我假设它是由于在混音器列表中的第一个ISampleSource引用和从另一个引用一次读取相同的流（并向前移动其位置）。总而言之，我不能使用相同的ISampleSource直接播放它，因为它是有状态的。
好的，所以我的第二种方法是使用Position来跟踪特定混音器列表条目何时被请求Read（）。我的意思是，当我将两个相同的ISampleSource添加到混音器列表时，它会为每个列表条目保留有关Position的其他数据。因此，当要求读取它时，将相同的ISampleSource给混音器两次通过要混合的源列表，它首先将其位置设置为上次为此列表条目完成的位置，然后读取此源。因此，即使我使用相同的源两次，其位置也会针对混音器列表中的每个注册单独处理。太棒了，我实际上得到了我的预期 - 同样的声音在同一时间（并行）播放了几次但是它不是很清楚 - 我在输出中有一些噼啪声，就像我有物理电缆缺陷一样。即使混合和播放单个声音也存在问题。我发现当我注释掉设置ISampleSource.Position的行时，问题就消失了。所以我认为寻找流是一个问题。不幸的是，我不确定为什么这是一个问题。我在SampleSource / WaveSource的实现中看到，它的Position实现将后备流位置与块大小对齐对齐。也许这就是原因，但我不确定。
我的下一个想法是实现＆＃34; SharedMemoryStream＆＃34;这就像MemoryStream，但能够为这样的流的几个实例提供相同的实际内存缓冲区。然后它将自己跟踪它的位置（以避免寻找我有问题），同时只有一个加载声音的内存表示。我不确定它是否会像预期的那样工作（很快就会尝试）但是我的一些实验告诉我它不会那么内存和CPU效率 - 当使用WaveFileReader时它会在内部创建一些消耗相当多的WaveFileChunk记忆和建造时间。因此，当使用＆＃34; SharedMemoryStream＆＃34;时，我仍然会看到一些其他问题。使用单个缓冲区的单独流，因为每个流需要一个WaveFileReader，并且每个Play请求都需要这样一个WaveFileReader（将混合中的ISampleSource排入队列）。
我在这里做的事情显然是错误的，只有新手会这样做吗？
PS。很抱歉对我的方法和实验进行了详尽的阐述。我只想清楚我对CSCore和音频处理的理解。我准备在我的问题/描述中删除不必要的部分。
更新：添加了最少的代码示例，让我重现问题。
以下是我遇到问题的代码的最小示例。
// Program.cs
using CSCore;
using CSCore.Codecs;
using CSCore.SoundOut;
using CSCore.Streams;
namespace AudioProblem
{
internal static class Program
{
private static void Main(string[] args)
{
var soundOut = new WasapiOut();
var soundMixer = new SoundMixer();
var sound = LoadSound("Heroic Demise (New).mp3");
soundOut.Initialize(soundMixer.ToWaveSource());
soundOut.Play();
soundMixer.AddSound(sound);
// Use the same sample source to have the same sound in play after 5 seconds.
// So two sounds are playing at the same time but are phase shifted by 5 seconds.
//Thread.Sleep(TimeSpan.FromSeconds(5));
//soundMixer.AddSound(sound);
}
private static ISampleSource LoadSound(string filePath)
{
var waveFileReader = CodecFactory.Instance.GetCodec(filePath);
return new CachedSoundSource(waveFileReader).ToSampleSource();
}
}
}
// SoundMixer.cs
using System;
using System.Collections.Generic;
using CSCore;
namespace AudioProblem
{
internal class SoundMixer : ISampleSource
{
private readonly object _lock = new object();
private readonly List<SoundSource> _soundSources = new List<SoundSource>();
private float[] _internalBuffer;
public SoundMixer()
{
var sampleRate = 44100;
var bits = 32;
var channels = 2;
var audioEncoding = AudioEncoding.IeeeFloat;
WaveFormat = new WaveFormat(sampleRate, bits, channels, audioEncoding);
}
public int Read(float[] buffer, int offset, int count)
{
var numberOfSamplesStoredInBuffer = 0;
if (count > 0 && _soundSources.Count > 0)
lock (_lock)
{
Array.Clear(buffer, offset, count);
_internalBuffer = _internalBuffer.CheckBuffer(count);
for (var i = _soundSources.Count - 1; i >= 0; i--)
{
var soundSource = _soundSources[i];
// Here is the magic. Look at Read implementation.
soundSource.Read(_internalBuffer, count);
for (int j = offset, k = 0; k < soundSource.SamplesRead; j++, k++)
{
buffer[j] += _internalBuffer[k];
}
if (soundSource.SamplesRead > numberOfSamplesStoredInBuffer)
numberOfSamplesStoredInBuffer = soundSource.SamplesRead;
if (soundSource.SamplesRead == 0) _soundSources.Remove(soundSource);
}
}
return count;
}
public void Dispose()
{
throw new NotImplementedException();
}
public bool CanSeek => false;
public WaveFormat WaveFormat { get; }
public long Position
{
get => 0;
set => throw new NotSupportedException($"{nameof(SoundMixer)} does not support setting the {nameof(Position)}.");
}
public long Length => 0;
public void AddSound(ISampleSource sound)
{
lock (_lock)
{
_soundSources.Add(new SoundSource(sound));
}
}
private class SoundSource
{
private readonly ISampleSource _sound;
private long _position;
public SoundSource(ISampleSource sound)
{
_sound = sound;
}
public int SamplesRead { get; private set; }
public void Read(float[] buffer, int count)
{
// Set last remembered position (initially 0).
// If this line is commented out, sound in my headphones is clear. But with this line it is crackling.
// If this line is commented out, if two SoundSource use the same ISampleSource output is buggy,
// but if line is present those are playing correctly but with crackling.
_sound.Position = _position;
// Read count of new samples.
SamplesRead = _sound.Read(buffer, 0, count);
// Remember position to be able to continue from where this SoundSource has finished last time.
_position = _sound.Position;
}
}
}
}
更新2：我找到了适合我的解决方案 - 详见下文。
看起来我对这个问题的初步想法是完全正确的，但并非100％确认。我已经介绍了一个在SoundSource的Read实现中执行的计数器，用于计算总共读取了多少样本来播放整个声音文件。我得到的是当我直接播放流时的情况的不同值，以及我在每次读取呼叫中保存并恢复位置时的另一种情况。对于后者，我计算的样本数量多于实际的声音文件，因此我假设由于这些过载样本而出现了一些噼啪声。我认为ISampleSource级别的位置会受到这个问题的影响，因为它将位置与其内部块大小对齐，因此这个属性似乎不足以停止并继续达到该精度级别。
所以我用＆＃34; SharedMemoryStream＆＃34;尝试了这个想法。看看在较低层次上管理储蓄和恢复仓位是否有效。它似乎做得很好。我最初担心使用这种方法创建大量WaveSource / SampleSource似乎并不是一个问题 - 我做了一些简单的测试，它的CPU和内存开销非常低。
以下是我实施此方法的代码，如果某些内容不明确或可以更好地完成，或者从一开始就应该以其他方式完成，请告诉我。
// Program.cs
using System;
using System.IO;
using System.Threading;
using CSCore;
using CSCore.Codecs.WAV;
using CSCore.SoundOut;
namespace AudioProblem
{
internal static class Program
{
private static void Main(string[] args)
{
var soundOut = new WasapiOut();
var soundMixer = new SoundMixer();
var sound = LoadSound("Heroic Demise (New).wav");
soundOut.Initialize(soundMixer.ToWaveSource());
soundOut.Play();
// Play first from shallow copy of shared stream
soundMixer.AddSound(new WaveFileReader(sound.MakeShared()).ToSampleSource());
Thread.Sleep(TimeSpan.FromSeconds(5));
// Play second from another shallow copy of shared stream
soundMixer.AddSound(new WaveFileReader(sound.MakeShared()).ToSampleSource());
Thread.Sleep(TimeSpan.FromSeconds(5));
soundOut.Stop();
}
private static SharedMemoryStream LoadSound(string filePath)
{
return new SharedMemoryStream(File.ReadAllBytes(filePath));
}
}
}
// SoundMixer.cs
using System;
using System.Collections.Generic;
using CSCore;
namespace AudioProblem
{
internal class SoundMixer : ISampleSource
{
private readonly List<SoundSource> _soundSources = new List<SoundSource>();
private readonly object _soundSourcesLock = new object();
private bool _disposed;
private float[] _internalBuffer;
public SoundMixer()
{
var sampleRate = 44100;
var bits = 32;
var channels = 2;
var audioEncoding = AudioEncoding.IeeeFloat;
WaveFormat = new WaveFormat(sampleRate, bits, channels, audioEncoding);
}
public int Read(float[] buffer, int offset, int count)
{
var numberOfSamplesStoredInBuffer = 0;
Array.Clear(buffer, offset, count);
lock (_soundSourcesLock)
{
CheckIfDisposed();
if (count > 0 && _soundSources.Count > 0)
{
_internalBuffer = _internalBuffer.CheckBuffer(count);
for (var i = _soundSources.Count - 1; i >= 0; i--)
{
var soundSource = _soundSources[i];
soundSource.Read(_internalBuffer, count);
for (int j = offset, k = 0; k < soundSource.SamplesRead; j++, k++)
{
buffer[j] += _internalBuffer[k];
}
if (soundSource.SamplesRead > numberOfSamplesStoredInBuffer)
numberOfSamplesStoredInBuffer = soundSource.SamplesRead;
if (soundSource.SamplesRead == 0)
{
_soundSources.Remove(soundSource);
soundSource.Dispose();
}
}
// TODO Normalize!
}
}
return count;
}
public void Dispose()
{
lock (_soundSourcesLock)
{
_disposed = true;
foreach (var soundSource in _soundSources)
{
soundSource.Dispose();
}
_soundSources.Clear();
}
}
public bool CanSeek => !_disposed;
public WaveFormat WaveFormat { get; }
public long Position
{
get
{
CheckIfDisposed();
return 0;
}
set => throw new NotSupportedException($"{nameof(SoundMixer)} does not support seeking.");
}
public long Length
{
get
{
CheckIfDisposed();
return 0;
}
}
public void AddSound(ISampleSource sound)
{
lock (_soundSourcesLock)
{
CheckIfDisposed();
// TODO Check wave format compatibility?
_soundSources.Add(new SoundSource(sound));
}
}
private void CheckIfDisposed()
{
if (_disposed) throw new ObjectDisposedException(nameof(SoundMixer));
}
private class SoundSource : IDisposable
{
private readonly ISampleSource _sound;
public SoundSource(ISampleSource sound)
{
_sound = sound;
}
public int SamplesRead { get; private set; }
public void Dispose()
{
_sound.Dispose();
}
public void Read(float[] buffer, int count)
{
SamplesRead = _sound.Read(buffer, 0, count);
}
}
}
}
// SharedMemoryStream.cs
using System;
using System.IO;
namespace AudioProblem
{
internal sealed class SharedMemoryStream : Stream
{
private readonly object _lock;
private readonly RefCounter _refCounter;
private readonly MemoryStream _sourceMemoryStream;
private bool _disposed;
private long _position;
public SharedMemoryStream(byte[] buffer) : this(new object(), new RefCounter(), new MemoryStream(buffer))
{
}
private SharedMemoryStream(object @lock, RefCounter refCounter, MemoryStream sourceMemoryStream)
{
_lock = @lock;
lock (_lock)
{
_refCounter = refCounter;
_sourceMemoryStream = sourceMemoryStream;
_refCounter.Count++;
}
}
public override bool CanRead
{
get
{
lock (_lock)
{
return !_disposed;
}
}
}
public override bool CanSeek
{
get
{
lock (_lock)
{
return !_disposed;
}
}
}
public override bool CanWrite => false;
public override long Length
{
get
{
lock (_lock)
{
CheckIfDisposed();
return _sourceMemoryStream.Length;
}
}
}
public override long Position
{
get
{
lock (_lock)
{
CheckIfDisposed();
return _position;
}
}
set
{
lock (_lock)
{
CheckIfDisposed();
_position = value;
}
}
}
// Creates another shallow copy of stream that uses the same underlying MemoryStream
public SharedMemoryStream MakeShared()
{
lock (_lock)
{
CheckIfDisposed();
return new SharedMemoryStream(_lock, _refCounter, _sourceMemoryStream);
}
}
public override void Flush()
{
}
public override long Seek(long offset, SeekOrigin origin)
{
lock (_lock)
{
CheckIfDisposed();
_sourceMemoryStream.Position = Position;
var seek = _sourceMemoryStream.Seek(offset, origin);
Position = _sourceMemoryStream.Position;
return seek;
}
}
public override void SetLength(long value)
{
throw new NotSupportedException($"{nameof(SharedMemoryStream)} is read only stream.");
}
// Uses position that is unique for each copy of shared stream
// to read underlying MemoryStream that is common for all shared copies
public override int Read(byte[] buffer, int offset, int count)
{
lock (_lock)
{
CheckIfDisposed();
_sourceMemoryStream.Position = Position;
var read = _sourceMemoryStream.Read(buffer, offset, count);
Position = _sourceMemoryStream.Position;
return read;
}
}
public override void Write(byte[] buffer, int offset, int count)
{
throw new NotSupportedException($"{nameof(SharedMemoryStream)} is read only stream.");
}
// Reference counting to dispose underlying MemoryStream when all shared copies are disposed
protected override void Dispose(bool disposing)
{
lock (_lock)
{
if (disposing)
{
_disposed = true;
_refCounter.Count--;
if (_refCounter.Count == 0) _sourceMemoryStream?.Dispose();
}
base.Dispose(disposing);
}
}
private void CheckIfDisposed()
{
if (_disposed) throw new ObjectDisposedException(nameof(SharedMemoryStream));
}
private class RefCounter
{
public int Count;
}
}
}