WCF在自托管服务上流式传输大数据(500MB / 1GB)

时间:2013-01-23 12:34:47

标签: c# .net wcf wcf-data-services

我目前在尝试使用WCF自托管服务(无IIS)发送大数据时遇到问题。 使用流式传输结果传输500MB,我的服务因System.OutOfMemoryException而崩溃。 是否有可能传输这么多数据?

这是我的WCF配置:

<system.serviceModel>
<services>
  <service  name="CIService" behaviorConfiguration="CIBehavior">        
    <host>
      <baseAddresses>
        <add baseAddress="net.tcp://localhost:6547/CIService/CIService.svc" />
      </baseAddresses>
    </host>
    <endpoint binding="netTcpBinding" 
        bindingConfiguration="netTcpBindingConfig" 
        behaviorConfiguration="CIBehavior.EndpointBehavior" 
        contract="CIService.ICreatable" />
    <endpoint address="mex" 
        binding="mexHttpBinding" 
        name="mexTcpBinding" 
        contract="IMetadataExchange" />
  </service>
</services>
<serviceHostingEnvironment multippleSiteBindingEnabled="True" />
<bindings>
  <netTcpBinding>
    <binding name="netTcpBindingConfig" closeTimeout="00:01:00" openTimeout="00:01:00" 
        receiveTimeout="01:00:00" sendTimeout="00:10:00" 
        hostNameComparisonMode="StrongWildcard" listenBacklog="10" maxConnections="10"
        maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647" 
        transferMode="Streamed">
      <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647"
        maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
    </binding>
  </netTcpBinding>
</bindings>

<behaviors>
  <serviceBehaviors>
    <behavior name="CIBehavior">
      <serviceMetadata httpGetEnabled="False" />
      <serviceDebug includeExceptionDetailInFaults="true" />
      <serviceThrottling maxConcurrentCalls="200"  maxConcurrentInstances="2147483647" maxConcurrentSessions="100" />
      <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
    </behavior>
  </serviceBehaviors>
  <endpointBehavior>
    <behavior name="CIBehavior.EndpointBehavior">
      <dataContractSerializer maxItemsInObjectGraph="2147483647" />
    </behavior>
  </endpointBehavior>
</behaviors>
</system.serviceModel>

我的客户端配置:

<system.serviceModel>
<bindings>
  <netTcpBinding>
    <binding name="NetTcpBinding_ICreatable" 
             closeTimeout="00:01:00" openTimeout="00:01:00" 
             receiveTimeout="01:00:00" sendTimeout="00:10:00" 
             transactionFlow="false" 
             transferMode="Streamed" 
             transactionProtocol="OleTransactions" 
             hostNameComparisonMode="StrongWildcard" 
             listenBacklog="10" 
             maxBufferPoolSize="2147483647" 
             maxBufferSize="2147483647" 
             maxConnections="10"
             maxReceivedMessageSize ="2147483647">
      <readerQuotas
        maxDepth="2147483647" 
        maxStringContentLength="2147483647" 
        maxArrayLength="2147483647" 
        maxBytesPerRead="2147483647" 
        maxNameTableCharCount="2147483647" />
      <reliableSession ordered="true" inactivityTimeout="00:10:00" anabled="false" />
    </binding>
  </netTcpBinding>
</bindings>
<client>
  <endpoint name="NetTcpBinding_ICreatable" 
      address="net.tcp://localhost:6547/CIService/CIService.svc" 
      binding="netTcpBinding" 
      bindingConfiguration="NetTcpBinding_ICreatable" 
      behaviorConfiguration="CIBehavior.EndpointBehavior" 
      contract="ICreatable" />
</client>
<behaviors>
  <endpointBehavior>
    <behavior name="CIBehavior.EndpointBehavior">
      <dataContractSerializer maxItemsInObjectGraph="2147483647" />
    </behavior>
  </endpointBehavior>
</behaviors>
</system.serviceModel> 

1 个答案:

答案 0 :(得分:44)

您不需要 maxBufferSize maxBufferPoolSize 设置如此之高,这些可能会导致您的内存不足异常。默认值应该没问题。

在MSDN上查看Large Data and Streaming,特别是大数据的特殊安全注意事项这一部分内容很重要

  

需要MaxBufferSize属性来约束内存   WCF缓冲区。将其设置为安全值(或保留它)非常重要   流式传输时的默认值)例如,假设你的   服务必须接收最大4 GB的文件并将其存储在   本地磁盘。假设你的记忆也受到限制   你一次只能缓冲64 KB的数据。然后你会设置   MaxReceivedMessageSize为4 GB,MaxBufferSize为64 KB。也,   在您的服务实现中,您必须确保只读   来自64 KB块的传入流,并且不读取下一个   上一个块之前的块已写入磁盘并被丢弃   来自记忆。

我汇总了一个非常简单的示例,将数据从自托管服务传输到控制台客户端。为了保持帖子简短,我只添加了服务器代码和部分客户端。

服务合约

using System.IO;
using System.ServiceModel;

namespace Service
{
    [ServiceContract]
    public interface IStream
    {
        [OperationContract]
        Stream GetLargeObject();
    }
}

服务实施

using System;
using System.IO;
using System.ServiceModel;

namespace Service
{
   [ServiceBehavior]
   public class StreamService : IStream
   {
       public Stream GetLargeObject()
       {
           // Add path to a big file, this one is 2.5 gb
           string filePath = Path.Combine(Environment.CurrentDirectory, "C:\\Temp\\BFBC2_PC_Client_R11_795745_Patch.exe");

        try
        {
            FileStream imageFile = File.OpenRead(filePath);
            return imageFile;
        }
        catch (IOException ex)
        {
            Console.WriteLine(String.Format("An exception was thrown while trying to open file {0}", filePath));
            Console.WriteLine("Exception is: ");
            Console.WriteLine(ex.ToString());
            throw;
        }
    }
 }
}

服务主要

using System;
using System.ServiceModel;

namespace Service
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                using (var serviceHost = new ServiceHost(typeof(StreamService)))
                {
                    serviceHost.Open();

                    Console.WriteLine("Press Any Key to end");
                    Console.ReadKey();
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
        }
    }
}

服务app.config

<?xml version="1.0"?>
<configuration>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
  </startup>

  <system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name="StreamServiceBehavior">
          <serviceMetadata httpGetEnabled="True" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <bindings>
      <netTcpBinding>
        <binding name="NewBinding0" transferMode="Streamed"/>
      </netTcpBinding>
    </bindings>
    <services>
      <service behaviorConfiguration="StreamServiceBehavior" name="Service.StreamService">
        <endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
          bindingConfiguration="NewBinding0" bindingName="" contract="Service.IStream" />
        <endpoint address="mex" binding="mexHttpBinding"
          contract="IMetadataExchange" />
        <host>
          <baseAddresses>
            <add baseAddress="http://localhost:8080/StreamService" />
          </baseAddresses>
        </host>
      </service>
    </services>
  </system.serviceModel>
</configuration>

启动服务,可能需要在管理员帐户下运行才能打开套接字。创建客户端控制台应用程序并使用URL http:// localhost:8080 / StreamService添加服务引用,使用Service作为生成的客户端的命名空间。

客户主要

using System;
using System.IO;
using Client.Service;

namespace Client
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                using (StreamClient streamClient = new StreamClient())
                {
                    streamClient.Open();

                    using (FileStream fileStream = new FileStream("c:\\temp\\bigfile.exe",FileMode.Create))
                    {
                        streamClient.GetLargeObject().CopyTo(fileStream);    
                    }
                }

                Console.WriteLine("Press any key to end");
                Console.ReadKey();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }
}

生成的客户端配置文件需要稍微修改,增加 receiveTimeout 并设置 maxReceivedMessageSize =“4294967295”

<system.serviceModel>
    <bindings>
        <netTcpBinding>
            <binding name="NetTcpBinding_IStream" closeTimeout="00:01:00"
                openTimeout="00:01:00" receiveTimeout="00:30:00" sendTimeout="00:01:00"
                transactionFlow="false" transferMode="Streamed" transactionProtocol="OleTransactions"
                hostNameComparisonMode="StrongWildcard" listenBacklog="10"
                maxBufferPoolSize="524288" maxBufferSize="65536" maxConnections="10"
                maxReceivedMessageSize="4294967295">
                <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                    maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                <reliableSession ordered="true" inactivityTimeout="00:10:00"
                    enabled="false" />
                <security mode="Transport">
                    <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
                    <message clientCredentialType="Windows" />
                </security>
            </binding>
        </netTcpBinding>
    </bindings>
    <client>
        <endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
            bindingConfiguration="NetTcpBinding_IStream" contract="Service.IStream"
            name="NetTcpBinding_IStream">

        </endpoint>
    </client>
</system.serviceModel>

启动服务然后启动客户端。它会毫无问题地流式传输大文件。