MF SinkWriter写入示例失败

时间：2017-06-07 03:15:34

标签： c# directx ms-media-foundation sharpdx dxgi

我尝试使用MediaFoundation将ID3D11Texture2D编码为mp4。以下是我目前的代码。

初始化接收器编写器

private int InitializeSinkWriter(String outputFile, int videoWidth, int videoHeight)
    {
        IMFMediaType mediaTypeIn = null;
        IMFMediaType mediaTypeOut = null;
        IMFAttributes attributes = null;

        int hr = 0;

        if (Succeeded(hr)) hr = (int)MFExtern.MFCreateAttributes(out attributes, 1);
        if (Succeeded(hr)) hr = (int)attributes.SetUINT32(MFAttributesClsid.MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, 1);            
        if (Succeeded(hr)) hr = (int)attributes.SetUINT32(MFAttributesClsid.MF_LOW_LATENCY, 1);

        // Create the sink writer 
        if (Succeeded(hr)) hr = (int)MFExtern.MFCreateSinkWriterFromURL(outputFile, null, attributes, out sinkWriter);

        // Create the output type
        if (Succeeded(hr)) hr = (int)MFExtern.MFCreateMediaType(out mediaTypeOut);
        if (Succeeded(hr)) hr = (int)mediaTypeOut.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Video);
        if (Succeeded(hr)) hr = (int)mediaTypeOut.SetGUID(MFAttributesClsid.MF_TRANSCODE_CONTAINERTYPE, MFTranscodeContainerType.MPEG4);
        if (Succeeded(hr)) hr = (int)mediaTypeOut.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, MFMediaType.H264);
        if (Succeeded(hr)) hr = (int)mediaTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AVG_BITRATE, videoBitRate);
        if (Succeeded(hr)) hr = (int)mediaTypeOut.SetUINT32(MFAttributesClsid.MF_MT_INTERLACE_MODE, (int)MFVideoInterlaceMode.Progressive);            

        if (Succeeded(hr)) hr = (int)MFExtern.MFSetAttributeSize(mediaTypeOut, MFAttributesClsid.MF_MT_FRAME_SIZE, videoWidth, videoHeight);
        if (Succeeded(hr)) hr = (int)MFExtern.MFSetAttributeRatio(mediaTypeOut, MFAttributesClsid.MF_MT_FRAME_RATE, VIDEO_FPS, 1);
        if (Succeeded(hr)) hr = (int)MFExtern.MFSetAttributeRatio(mediaTypeOut, MFAttributesClsid.MF_MT_PIXEL_ASPECT_RATIO, 1, 1);
        if (Succeeded(hr)) hr = (int)sinkWriter.AddStream(mediaTypeOut, out streamIndex);



        // Create the input type 
        if (Succeeded(hr)) hr = (int)MFExtern.MFCreateMediaType(out mediaTypeIn);
        if (Succeeded(hr)) hr = (int)mediaTypeIn.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Video);
        if (Succeeded(hr)) hr = (int)mediaTypeIn.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, MFMediaType.ARGB32);
        if (Succeeded(hr)) hr = (int)mediaTypeIn.SetUINT32(MFAttributesClsid.MF_SA_D3D11_AWARE, 1);
        if (Succeeded(hr)) hr = (int)mediaTypeIn.SetUINT32(MFAttributesClsid.MF_MT_INTERLACE_MODE, (int)MFVideoInterlaceMode.Progressive);
        if (Succeeded(hr)) hr = (int)MFExtern.MFSetAttributeSize(mediaTypeIn, MFAttributesClsid.MF_MT_FRAME_SIZE, videoWidth, videoHeight);
        if (Succeeded(hr)) hr = (int)MFExtern.MFSetAttributeRatio(mediaTypeIn, MFAttributesClsid.MF_MT_FRAME_RATE, VIDEO_FPS, 1);
        if (Succeeded(hr)) hr = (int)MFExtern.MFSetAttributeRatio(mediaTypeIn, MFAttributesClsid.MF_MT_PIXEL_ASPECT_RATIO, 1, 1);
        if (Succeeded(hr)) hr = (int)sinkWriter.SetInputMediaType(streamIndex, mediaTypeIn, null);


        // Start accepting data
        if (Succeeded(hr)) hr = (int)sinkWriter.BeginWriting();


        COMBase.SafeRelease(mediaTypeOut);
        COMBase.SafeRelease(mediaTypeIn);

        return hr;
    }

书写框架

 int hr = 0;
        IMFSample sample = null;
        IMFMediaBuffer buffer = null;
        IMF2DBuffer p2Dbuffer = null;
        object texNativeObject = Marshal.GetObjectForIUnknown(surface.NativePointer);

        if (Succeeded(hr)) hr = (int)MFExtern.MFCreateDXGISurfaceBuffer(new Guid("6f15aaf2-d208-4e89-9ab4-489535d34f9c"), texNativeObject, 0, false, out p2Dbuffer);

        buffer = MFVideoEncoderST.ReinterpretCast<IMF2DBuffer,IMFMediaBuffer>(p2Dbuffer);
        int length=0;
        if (Succeeded(hr)) hr = (int)p2Dbuffer.GetContiguousLength(out length);
        if (Succeeded(hr)) hr = (int)buffer.SetCurrentLength(length);


        if (Succeeded(hr)) hr = (int)MFExtern.MFCreateVideoSampleFromSurface(null, out sample);

        if (Succeeded(hr)) hr = (int)sample.AddBuffer(buffer);
        if (Succeeded(hr)) hr = (int)sample.SetSampleTime(prevRecordingDuration);
        if (Succeeded(hr)) hr = (int)sample.SetSampleDuration((recordDuration - prevRecordingDuration));

        if (Succeeded(hr)) hr = (int)sinkWriter.WriteSample(streamIndex, sample);


        COMBase.SafeRelease(sample);
        COMBase.SafeRelease(buffer);

使用MFTRACE我得到以下错误。

    02:48:04.99463 CMFSinkWriterDetours::WriteSample @024BEA18 Stream Index 0x0, Sample @17CEACE0, Time 571ms, Duration 16ms, Buffers 1, Size 4196352B,2088,2008 02:48:04.99465 CMFSinkWriterDetours::WriteSample @024BEA18 failed hr=0x887A0005 (null)2088,2008 
02:48:05.01090 CMFSinkWriterDetours::WriteSample @024BEA18 Stream Index 0x0, Sample @17CE9FC0, Time 587ms, Duration 17ms, Buffers 1, Size 4196352B,2088,2008 02:48:05.01091 CMFSinkWriterDetours::WriteSample @024BEA18 failed hr=0x887A0005 (null)2088,2008 
02:48:05.02712 CMFSinkWriterDetours::WriteSample @024BEA18 Stream Index 0x0, Sample @17CEACE0, Time 604ms, Duration 16ms, Buffers 1, Size 4196352B,2088,2008 02:48:05.02713 CMFSinkWriterDetours::WriteSample @024BEA18 failed hr=0x887A0005 (null)

有谁能告诉我我的代码有什么问题？我只能生成0字节的mp4文件。

2 个答案:

答案 0 :(得分：3)

这里有一些潜在的问题。罗曼提到了两个大的，所以我将详细说明这些。我还有一些其他批评/建议。

未使用IMFDXGIDeviceManager

要在Media Foundation中使用硬件加速，您需要创建一个DirectX设备管理器对象，DX9为IDirect3DDeviceManager9，DXGI为IMFDXGIDeviceManager。我强烈建议阅读该界面的所有MSDN文档。这是必要的原因是因为必须在所使用的所有协作硬件MF变换上共享相同的DX设备，因为它们都需要访问设备控制的共享GPU存储器，并且每个设备都需要对设备进行独占控制。 39;工作，所以需要一个锁定系统。设备管理器对象提供锁定系统，也是向一个或多个转换提供DX设备的标准方法。对于DXGI，您可以使用MFCreateDXGIDeviceManager创建此内容。

从那里，您需要创建DX11设备，并使用DX11设备呼叫IMFDXGIDeviceManager::ResetDevice。然后，您需要为Sink Writer本身设置设备管理器，这在上面提供的代码中没有完成。这样就完成了：

// ... inside your InitializeSinkWriter function that you listed above

// I'm assuming you've already created and set up the DXGI device manager elsewhere
IMFDXGIDeviceManager pDeviceManager;

// Passing 3 as the argument because we're adding 3 attributes immediately, saves re-allocations
if (Succeeded(hr)) hr = (int)MFExtern.MFCreateAttributes(out attributes, 3);
if (Succeeded(hr)) hr = (int)attributes.SetUINT32(MFAttributesClsid.MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, 1);            
if (Succeeded(hr)) hr = (int)attributes.SetUINT32(MFAttributesClsid.MF_LOW_LATENCY, 1);

// Here's the key piece!
if (Succeeded(hr)) hr = (int)attributes.SetUnknown(MFAttributesClsid.MF_SINK_WRITER_D3D_MANAGER, pDeviceManager);

// Create the sink writer 
if (Succeeded(hr)) hr = (int)MFExtern.MFCreateSinkWriterFromURL(outputFile, null, attributes, out sinkWriter);

这实际上将启用D3D11对硬件编码器的支持，并允许其访问以读取您传入的Texture2D。值得注意的是MF_SINK_WRITER_D3D_MANAGER适用于两个DX9和DXGI设备管理员。

<小时/>

编码器缓冲相同纹理的多个IMFSample实例

这也是导致问题的一个潜在原因 - 即使不是明显问题的原因，它至少会导致许多意外行为。根据Roman的评论，许多编码器将缓冲多个帧作为其编码过程的一部分。使用Sink Writer时，您不会看到这种行为，因为它会为您处理所有细节。但是，您要完成的任务（即将D3D11纹理作为输入帧发送）的水平足够低，您开始不必担心Sink Writer正在使用的编码器MFT的内部细节。

大多数视频编码器MFT将使用某种大小的内部缓冲区来存储通过IMFTransform::ProcessInput提供的最后 N 样本。这具有副作用，即在生成任何输出之前必须提供多个样本作为输入。视频编码器需要按顺序访问多个样本，因为它们使用后续帧来确定如何编码当前帧。换句话说，如果解码器正在处理帧0，则可能还需要查看帧1,2和3。从技术角度来看，这是因为inter-frame prediction和运动估计之类的东西。一旦编码器处理完最旧的样本，它就会生成一个输出缓冲区（另一个IMFSample对象，但这次在输出端通过IMFTransform::ProcessOutput）然后丢弃它正在处理的输入样本（通过调用IUnknown::Release），然后请求更多输入，并最终移动到下一帧。您可以在MSDN文章Processing Data in the Encoder

中阅读有关此过程的更多信息

正如罗马所提到的，这意味着你将ID3D11Texture2D封装在IMFMediaBuffer内的IMFSample内，然后将其传递给Sink Writer。作为编码过程的一部分，编码器可能会缓冲该样本。当编码器工作时，Texture2D的内容可能会发生变化，这可能会导致各种问题。即使这不会导致程序错误，它肯定会导致非常奇怪的编码视频输出。想象一下，如果编码器试图预测下一帧中一帧的可视内容如何变化，那么两帧的实际可视内容将从编码器下更新！

这个特定的问题正在发生，因为编码器只有一个指向你的IMFSample实例的指针，它最终只是你ID3D11Texture2D对象的指针，而且该对象是一种指针引用可变图形内存。最终，由于程序的其他部分，图形内存的内容正在发生变化，但由于它总是更新相同的GPU纹理，因此发送编码器的每个样本都指向相同的单个纹理。这意味着无论何时更新纹理，通过更改GPU内存，所有活动的IMFSample对象都将反映这些更改，因为它们都有效地指向相同的GPU纹理。

要解决此问题，您需要分配多个ID3D11Texture2D个对象，以便在将其提供给Sink Writer时，可以将一个纹理与一个IMFSample配对。这将通过使每个样本指向唯一纹理来解决所有样本指向相同单个GPU纹理的问题。但是，您不必知道需要创建多少纹理，因此最安全的方法是编写自己的纹理分配器。这仍然可以在C＃中完成它的价值，MediaFoundation.NET已经定义了你需要使用的接口。

分配器应该保留一个＆＃34; free＆＃34; SharpDX.Texture2D个对象 - Sink Writer / encoder当前未使用的对象。您的程序应该能够从分配器请求新的纹理对象，在这种情况下，它将从空闲列表返回一个对象，或者创建一个新的纹理来容纳请求。

下一个问题是知道编码器何时丢弃IMFSample对象，以便您可以将附加的纹理添加回空闲列表。碰巧，您当前使用的MFCreateVideoSampleFromSurface函数会分配实现IMFTrackedSample接口的示例。您需要该接口，以便在释放样本时收到通知，以便您可以回收Texture2D个对象。

诀窍是你必须告诉样本你是分配器。首先，您的allocator类需要实现IMFAsyncCallback。如果通过IMFTrackedSample::SetAllocator在样本上设置allocator类，则会调用分配器的IMFAsyncCallback::Invoke方法，并在编码器释放样本时将IMFAsyncResult作为参数传递。这是分配器类的外观的一般示例。 

sealed class TextureAllocator : IMFAsyncCallback, IDisposable
{
    private ConcurrentStack<SharpDX.Direct3D11.Texture2D> m_freeStack;
    private static readonly Guid s_IID_ID3D11Texture2D = new Guid("6f15aaf2-d208-4e89-9ab4-489535d34f9c");

    // If all textures are the exact same size and color format,
    // consider making those parameters private class members and
    // requiring they be specified as arguments to the constructor.
    public TextureAllocator()
    {
        m_freeStack = new ConcurrentStack<SharpDX.Direct3D11.Texture2D>();
    }

    private bool disposedValue = false;
    private void Dispose(bool disposing)
    {
        if(!disposedValue)
        {
            if(disposing)
            {
                // Dispose managed resources here
            }

            if(m_freeStack != null)
            {
                SharpDX.Direct3D11.Texture2D texture;
                while(m_freeStack.TryPop(out texture))
                {
                    texture.Dispose();
                }
                m_freeStack = null;
            }

            disposedValue = true;
        }
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    ~TextureAllocator()
    {
        Dispose(false);
    }

    private SharpDX.Direct3D11.Texture2D InternalAllocateNewTexture()
    {
        // Allocate a new texture with your format, size, etc here.
    }

    public SharpDX.Direct3D11.Texture2D AllocateTexture()
    {
        SharpDX.Direct3D11.Texture2D existingTexture;
        if(m_freeStack.TryPop(out existingTexture))
        {
            return existingTexture;
        }
        else
        {
            return InternalAllocateNewTexture();
        }
    }

    public IMFSample CreateSampleAndAllocateTexture()
    {
        IMFSample pSample;
        IMFTrackedSample pTrackedSample;
        HResult hr;

        // Create the video sample. This function returns an IMFTrackedSample per MSDN
        hr = MFExtern.MFCreateVideoSampleFromSurface(null, out pSample);
        MFError.ThrowExceptionForHR(hr);

        // Query the IMFSample to see if it implements IMFTrackedSample
        pTrackedSample = pSample as IMFTrackedSample;
        if(pTrackedSample == null)
        {
            // Throw an exception if we didn't get an IMFTrackedSample
            // but this shouldn't happen in practice.
            throw new InvalidCastException("MFCreateVideoSampleFromSurface returned a sample that did not implement IMFTrackedSample");
        }

        // Use our own class to allocate a texture
        SharpDX.Direct3D11.Texture2D availableTexture = AllocateTexture();
        // Convert the texture's native ID3D11Texture2D pointer into
        // an IUnknown (represented as as System.Object)
        object texNativeObject = Marshal.GetObjectForIUnknown(availableTexture.NativePointer);

        // Create the media buffer from the texture
        IMFMediaBuffer p2DBuffer;
        hr = MFExtern.MFCreateDXGISurfaceBuffer(s_IID_ID3D11Texture2D, texNativeObject, 0, false, out p2DBuffer);
        // Release the object-as-IUnknown we created above
        COMBase.SafeRelease(texNativeObject);
        // If media buffer creation failed, throw an exception
        MFError.ThrowExceptionForHR(hr);

        // Set the owning instance of this class as the allocator
        // for IMFTrackedSample to notify when the sample is released
        pTrackedSample.SetAllocator(this, null);

        // Attach the created buffer to the sample
        pTrackedSample.AddBuffer(p2DBuffer);

        return pTrackedSample;
    }

    // This is public so any textures you allocate but don't make IMFSamples 
    // out of can be returned to the allocator manually.
    public void ReturnFreeTexture(SharpDX.Direct3D11.Texture2D freeTexture)
    {
        m_freeStack.Push(freeTexture);
    }

    // IMFAsyncCallback.GetParameters
    // This is allowed to return E_NOTIMPL as a way of specifying
    // there are no special parameters.
    public HResult GetParameters(out MFAsync pdwFlags, out MFAsyncCallbackQueue pdwQueue)
    {
        pdwFlags = MFAsync.None;
        pdwQueue = MFAsyncCallbackQueue.Standard;
        return HResult.E_NOTIMPL;
    }

    public HResult Invoke(IMFAsyncResult pResult)
    {
        object pUnkObject;
        IMFSample pSample = null;
        IMFMediaBuffer pBuffer = null;
        IMFDXGIBuffer pDXGIBuffer = null;

        // Get the IUnknown out of the IMFAsyncResult if there is one
        HResult hr = pResult.GetObject(out pUnkObject);
        if(Succeeded(hr))
        {
            pSample = pUnkObject as IMFSample;
        }

        if(pSample != null)
        {
            // Based on your implementation, there should only be one 
            // buffer attached to one sample, so we can always grab the
            // first buffer. You could add some error checking here to make
            // sure the sample has a buffer count that is 1.
            hr = pSample.GetBufferByIndex(0, out pBuffer);
        }

        if(Succeeded(hr))
        {
            // Query the IMFMediaBuffer to see if it implements IMFDXGIBuffer
            pDXGIBuffer = pBuffer as IMFDXGIBuffer;
        }

        if(pDXGIBuffer != null)
        {
           // Got an IMFDXGIBuffer, so we can extract the internal 
           // ID3D11Texture2D and make a new SharpDX.Texture2D wrapper.
           hr = pDXGIBuffer.GetResource(s_IID_ID3D11Texture2D, out pUnkObject);
        }

        if(Succeeded(hr))
        {
           // If we got here, pUnkObject is the native D3D11 Texture2D as
           // a System.Object, but it's unlikely you have an interface 
           // definition for ID3D11Texture2D handy, so we can't just cast
           // the object to the proper interface.

           // Happily, SharpDX supports wrapping System.Object within
           // SharpDX.ComObject which makes things pretty easy.
           SharpDX.ComObject comWrapper = new SharpDX.ComObject(pUnkObject);

           // If this doesn't work, or you're using something like SlimDX
           // which doesn't support object wrapping the same way, the below
           // code is an alternative way.
           /*
           IntPtr pD3DTexture2D = Marshal.GetIUnknownForObject(pUnkObject);
           // Create your wrapper object here, like this for SharpDX
           SharpDX.ComObject comWrapper = new SharpDX.ComObject(pD3DTexture2D);
           // or like this for SlimDX
           SlimDX.Direct3D11.Texture2D.FromPointer(pD3DTexture2D);
           Marshal.Release(pD3DTexture2D);
           */

           // You might need to query comWrapper for a SharpDX.DXGI.Resource
           // first, then query that for the SharpDX.Direct3D11.Texture2D.
           SharpDX.Direct3D11.Texture2D texture = comWrapper.QueryInterface<SharpDX.Direct3D11.Texture2D>();
           if(texture != null)
           {
               // Now you can add "texture" back to the allocator's free list
               ReturnFreeTexture(texture);
           }
        }
    }
}

<小时/>

在Sink Writer输入媒体类型

上设置MF_SA_D3D_AWARE

我不认为这会导致您感到害羞HRESULT，但无论如何都不是正确的做法。 MF_SA_D3D_AWARE（及其DX11对应物MF_SA_D3D11_AWARE）是由IMFTransform对象设置的属性，用于通知您变换分别通过DX9或DX11支持图形加速。无需在Sink Writer的输入媒体类型上设置此项。

<小时/>

{h2> SafeRelease 上没有texNativeObject

我建议在COMBase.SafeRelease()上调用texNativeObject，否则可能会泄漏内存。那，或者你不必要地延长那个COM对象的生命周期，直到GC清理你的引用计数为止

<小时/>

不必要的施法

这是您上面代码的一部分：

buffer = MFVideoEncoderST.ReinterpretCast<IMF2DBuffer,IMFMediaBuffer>(p2Dbuffer);
int length=0;
if (Succeeded(hr)) hr = (int)p2Dbuffer.GetContiguousLength(out length);
if (Succeeded(hr)) hr = (int)buffer.SetCurrentLength(length);

我不确定你的ReinterpretCast函数在做什么，但如果你做需要在C＃中执行QueryInterface样式转换，你可以使用as运营商或常规演员。

// pMediaBuffer is of type IMFMediaBuffer and has been created elsewhere
IMF2DBuffer p2DBuffer = pMediaBuffer as IMF2DBuffer;
if(p2DBuffer != null)
{
    // pMediaBuffer is an IMFMediaBuffer that also implements IMF2DBuffer
}
else
{
    // pMediaBuffer does not implement IMF2DBuffer
}

答案 1 :(得分：1)

第一个问题：IMFDXGIDeviceManager::ResetDevice总是失败。

在我之前的回答评论中使用@kripto后，我们诊断出了许多其他问题。最大的问题是设置IMFDXGIDeviceManager以便使硬件H.264编码器MFT能够接受包含在IMFDXGIBuffer内的Direct3D11 Texture2D样本。代码中有一个非常难以察觉的错误： 

// pDevice is a SharpDX.Direct3D11.Texture2D instance
// pDevice.NativePointer is an IntPtr that refers to the native IDirect3D11Device
// being wrapped by SharpDX.
IMFDXGIDeviceManager pDeviceManager;
object d3dDevice = Marshal.GetIUnknownForObject(pDevice.NativePointer);
HResult hr = MFExtern.MFCreateDXGIDeviceManager(out resetToken, out pDeviceManager);
if(Succeeded(hr))
{
    // The signature of this is:
    // HResult ResetDevice(object d3d11device, int resetToken);
    hr = pDeviceManager.ResetDevice(d3dDevice, resetToken);
}

以上是上述代码中发生的情况。创建了设备管理器，但为了让编码器MFT访问Texture2D个样本，它需要创建纹理的相同Direct3D设备的副本。因此，您必须在设备管理器上调用IMFDXGIDeviceManager::ResetDevice才能为其提供Direct3D设备。有关ResetDevice的一些重要脚注，请参见[1]。 SharpDX仅提供对指向本机IntPtr的{​​{1}}的访问，但MediaFoundation.NET接口需要传入IDirect3D11Device。

看到错误了吗？上面的代码类型 - 检查和编译就好了，但包含一个严重错误。错误是使用Marshal.GetIUnknownForObject而不是Marshal.GetObjectForIUnknown。有趣的是，因为object可以装object就好了，你可以使用完全相反的编组功能，它仍然编译得很好。问题是我们正在尝试将IntPtr转换为IntPtr内的.NET RCW，这是MediaFoundation.NET期望的object。此错误导致ResetDevice返回ResetDevice而不是正常工作。

<小时/>

第二个问题：奇怪的编码器输出

第二个问题是英特尔快速同步视频H.264编码器MFT不是特别高兴，虽然它是正确创建的，但在生成的文件的开头还有一两个黑色输出，以及作为前几秒的阻塞和运动错误，有时一半的视频是灰色的，并且没有显示实际的重复桌面图像。

我想确保实际的E_INVALIDARG对象正确地发送到编码器，所以我编写了一个简单的类来将Direct3D 11 Texture2D转储到Texture2D文件。我已将其包含在其他需要它的人中 - 这需要SharpDX和MediaFoundation.NET才能工作，尽管您可以通过在循环中使用CopyMemory来消除MF依赖关系来解释不同的步幅。请注意，这仅设置为使用DXGI.Format.B8G8R8A8_UNorm格式的纹理。它可能适用于其他格式的纹理，但输出看起来很奇怪。 

.png

一旦我确认我在进入编码器的过程中有很好的图像，我发现代码在调用IMFSinkWriter::SendStreamTick之后但在发送第一个using System; using System.Drawing; namespace ScreenCapture { class Texture2DDownload : IDisposable { private SharpDX.Direct3D11.Device m_pDevice; private SharpDX.Direct3D11.Texture2D m_pDebugTexture; public Texture2DDownload(SharpDX.Direct3D11.Device pDevice) { m_pDevice = pDevice; } /// <summary> /// Compare all the relevant properties of the texture descriptions for both input textures. /// </summary> /// <param name="texSource">The source texture</param> /// <param name="texDest">The destination texture that will have the source data copied into it</param> /// <returns>true if the source texture can be copied to the destination, false if their descriptions are incompatible</returns> public static bool TextureCanBeCopied(SharpDX.Direct3D11.Texture2D texSource, SharpDX.Direct3D11.Texture2D texDest) { if(texSource.Description.ArraySize != texDest.Description.ArraySize) return false; if(texSource.Description.Format != texDest.Description.Format) return false; if(texSource.Description.Height != texDest.Description.Height) return false; if(texSource.Description.MipLevels != texDest.Description.MipLevels) return false; if(texSource.Description.SampleDescription.Count != texDest.Description.SampleDescription.Count) return false; if(texSource.Description.SampleDescription.Quality != texDest.Description.SampleDescription.Quality) return false; if(texSource.Description.Width != texDest.Description.Width) return false; return true; } /// <summary> /// Saves the contents of a <see cref="SharpDX.Direct3D11.Texture2D"/> to a file with name contained in <paramref name="filename"/> using the specified <see cref="System.Drawing.Imaging.ImageFormat"/>. /// </summary> /// <param name="texture">The <see cref="SharpDX.Direct3D11.Texture2D"/> containing the data to save.</param> /// <param name="filename">The filename on disk where the output image should be saved.</param> /// <param name="imageFormat">The format to use when saving the output file.</param> public void SaveTextureToFile(SharpDX.Direct3D11.Texture2D texture, string filename, System.Drawing.Imaging.ImageFormat imageFormat) { // If the existing debug texture doesn't exist, or the incoming texture is different than the existing debug texture... if(m_pDebugTexture == null || !TextureCanBeCopied(m_pDebugTexture, texture)) { // Dispose of any existing texture if(m_pDebugTexture != null) { m_pDebugTexture.Dispose(); } // Copy the original texture's description... SharpDX.Direct3D11.Texture2DDescription newDescription = texture.Description; // Then modify the parameters to create a CPU-readable staging texture newDescription.BindFlags = SharpDX.Direct3D11.BindFlags.None; newDescription.CpuAccessFlags = SharpDX.Direct3D11.CpuAccessFlags.Read; newDescription.OptionFlags = SharpDX.Direct3D11.ResourceOptionFlags.None; newDescription.Usage = SharpDX.Direct3D11.ResourceUsage.Staging; // Re-generate the debug texture by copying the new texture's description m_pDebugTexture = new SharpDX.Direct3D11.Texture2D(m_pDevice, newDescription); } // Copy the texture to our debug texture m_pDevice.ImmediateContext.CopyResource(texture, m_pDebugTexture); // Map the debug texture's resource 0 for read mode SharpDX.DataStream data; SharpDX.DataBox dbox = m_pDevice.ImmediateContext.MapSubresource(m_pDebugTexture, 0, 0, SharpDX.Direct3D11.MapMode.Read, SharpDX.Direct3D11.MapFlags.None, out data); // Create a bitmap that's the same size as the debug texture Bitmap b = new Bitmap(m_pDebugTexture.Description.Width, m_pDebugTexture.Description.Height, System.Drawing.Imaging.PixelFormat.Format32bppRgb); // Lock the bitmap data to get access to the native bitmap pointer System.Drawing.Imaging.BitmapData bd = b.LockBits(new Rectangle(0, 0, b.Width, b.Height), System.Drawing.Imaging.ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format32bppRgb); // Use the native pointers to do a native-to-native memory copy from the mapped subresource to the bitmap data // WARNING: This might totally blow up if you're using a different color format than B8G8R8A8_UNorm, I don't know how planar formats are structured as D3D textures! // // You can use Win32 CopyMemory to do the below copy if need be, but you have to do it in a loop to respect the Stride and RowPitch parameters in case the texture width // isn't on an aligned byte boundary. MediaFoundation.MFExtern.MFCopyImage(bd.Scan0, bd.Stride, dbox.DataPointer, dbox.RowPitch, bd.Width * 4, bd.Height); /// Unlock the bitmap b.UnlockBits(bd); // Unmap the subresource mapping, ignore the SharpDX.DataStream because we don't need it. m_pDevice.ImmediateContext.UnmapSubresource(m_pDebugTexture, 0); data = null; // Save the bitmap to the desired filename b.Save(filename, imageFormat); b.Dispose(); b = null; } #region IDisposable Support private bool disposedValue = false; // To detect redundant calls protected virtual void Dispose(bool disposing) { if(!disposedValue) { if(disposing) { } if(m_pDebugTexture != null) { m_pDebugTexture.Dispose(); } disposedValue = true; } } // TODO: override a finalizer only if Dispose(bool disposing) above has code to free unmanaged resources. ~Texture2DDownload() { // Do not change this code. Put cleanup code in Dispose(bool disposing) above. Dispose(false); } // This code added to correctly implement the disposable pattern. public void Dispose() { // Do not change this code. Put cleanup code in Dispose(bool disposing) above. Dispose(true); GC.SuppressFinalize(this); } #endregion } } 之前没有调用IMFSinkWriter::BeginWriting。初始样本的时间差也是非零，这导致初始黑色输出。为了解决这个问题，我添加了以下代码： 

IMFSample

通过向汇编写入器发送流标记，它确定// Existing code to set the sample time and duration // recordDuration is the current frame time in 100-nanosecond units // prevRecordingDuration is the frame time of the last frame in // 100-nanosecond units sample.SetSampleTime(recordDuration); sample.SetSampleDuration(recordDuration - prevRecordingDuration); // The fix is here: if(frames == 0) { sinkWriter.SendStreamTick(streamIndex, recordDuration); sample.SetUINT32(MFAttributesClsid.MFSampleExtension_Discontinuity, 1); } sinkWriter.WriteSample(streamIndex, sample); frames++; 中的任何值现在被视为输出视频流的时间= 0点。换句话说，一旦您调用recordDuration并传入帧时间戳，所有后续时间戳都会从中减去初始时间戳。这是您在输出文件中立即显示第一个样本框的方法。

此外，每当调用SetStreamTick时，直接提供给接收器编写器的示例必须在其属性上设置MFSampleExtension_Discontinuity SendStreamTick。这意味着发送的样本中存在间隙，并且传递给编码器的帧是该间隙之后的第一帧。这或多或少告诉编码器从样本中制作关键帧，这可以防止我之前看到的运动和阻挡效果。

<小时/>

结果

一旦实施了这些修复，我测试了应用程序并实现了1920x1080分辨率和60帧/秒输出的全屏捕获。比特率设置为4096 kbit。对于大多数工作负载，英特尔i7-4510U笔记本电脑CPU的CPU使用率介于2.5％和7％之间 - 极端运动量会使其达到约10％。通过SysInternals的Process Explorer的GPU利用率介于1％和2％之间。

<小时/> [1]我相信其中一些是来自Direct3D 9的遗留物，当多线程支持没有很好地内置到DirectX API中时，只要任何组件使用它就必须锁定设备（即解码器，渲染器） ，编码器）。使用D3D 9可以调用1但是再也不能使用自己的指针指向设备。相反，您必须在您自己的代码中调用ResetDevice和LockDevice来获取设备指针，因为MFT可能在同一时刻使用该设备。在Direct3D 11中，在MFT和控制应用程序中同时使用同一设备似乎没有问题 - 尽管如果发生任何随机崩溃，我建议您阅读大量有关UnlockDevice和{{1}的内容工作和实现这些以确保设备始终是专门控制的。

相关问题
最新问题