Question

我需要一个功能，可以通过外部服务提供商的POST通过HTTP上传将大数据文件上传到服务器。目前，我使用以下weblient函数，并且适用于较小的文件：

_byteReturn = await _webClient.UploadDataTaskAsync(_url, File.ReadAllBytes(@"c:\tmp\test.zip"));

但是在那种情况下，存在2GB边框的问题，并且函数ReadAllBytes（）将所有字节读取到内存中。当然，我可以使用其他网络功能

_byteReturn = await _webClient.UploadFileTaskAsync(_url, @"c:\tmp\test.zip"));

但是使用该功能，我从服务器收到HTTP错误400。：/ 因此它尝试使用我自己的代码进行上传。

using (WebClient _webClient = new WebClient())
{
    _webClient.Headers[HttpRequestHeader.UserAgent] = "Test";
    _webClient.Headers[HttpRequestHeader.CacheControl] = "no-cache";
    _webClient.Headers[HttpRequestHeader.Authorization] = string.Format("Basic {0}", Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes("Username") + ":" + ("Password" )));

    using (StreamWriter _output = new StreamWriter(await _webClient.OpenWriteTaskAsync(_url)))
    {

        _output.AutoFlush = true;


        using (FileStream _fileStream = new FileStream(@"c:\tmp\install.esd", FileMode.Open, FileAccess.Read))
        {
            _bytesRead = 0;
            _readByteBuffer = new byte[_bufferLength];
            _bytesToRead = _fileStream.Length;


#if DEBUG

            FileStream _testOutFileStream = new FileStream(_writeTestFileNameZip, FileMode.OpenOrCreate, FileAccess.ReadWrite);
#endif
            do
            {
                _fileStream.Seek(_bytesRead, SeekOrigin.Begin);
                _readCount = _fileStream.Read(_readByteBuffer, 0, _bufferLength);
                _output.Write(_readByteBuffer);

#if DEBUG
                if (_testOutFileStream != null)
                { 
                    _testOutFileStream.Write(_readByteBuffer, 0, _readCount);
                    _testOutFileStream.Flush();
                }
#endif
                _bytesRead += _readCount;
            }
            while (_readCount > 0);

#if DEBUG
            if (_testOutFileStream != null)
                _testOutFileStream.Dispose();
#endif
        }

        if (_output != null)
            _output.Close();
    }
}

重点是“ IT WORKS”。我在上传过程中没有错误，并且在上传完成后从服务器得到了正确的答案，但是上传就像地狱一样快。（在10秒内以10MBit / s的速度上传1GB）。

我认为这是一个缓存问题，但是我不确定。谁知道问题出在哪里？

Answer 1

您可以尝试下面的代码，让我知道它是否为您解决了问题，您需要重新配置变量/等...

string _username = string.Empty;
string _password = string.Empty;
string _url = "http://example.com";
string _writeTestFileNameZip = string.Empty;

using (WebClient _webClient = new WebClient())
{
    _webClient.Headers[HttpRequestHeader.UserAgent] = "Test";
    _webClient.Headers[HttpRequestHeader.CacheControl] = "no-cache";
    _webClient.Headers[HttpRequestHeader.Authorization] = $"{Convert.ToBase64String(Encoding.ASCII.GetBytes($"{_username}:{_password}"))}";


    using (Stream _output = await _webClient.OpenWriteTaskAsync(_url))
    using (FileStream _fileStream = new FileStream(@"c:\tmp\install.esd", FileMode.Open, FileAccess.Read))
    {
        byte[] _readByteBuffer = new byte[1024 * 4];
        long _bytesToRead = _fileStream.Length;
        int _bytesRead = 0;

        _fileStream.Seek(0, SeekOrigin.Begin);

        while ((_bytesRead = await _fileStream.ReadAsync(_readByteBuffer, 0, _readByteBuffer.Length)) > 0)
        {
            await _output.WriteAsync(_readByteBuffer, 0, _bytesRead);
        }
    }

}

Answer 2

基于Aydin Adn和其他搜索结果，我找到了最终的解决方案。必须将 AllowWriteStreamBuffering 属性设置为false。原始的webclient无法实现此功能，而只能在扩展类中实现。就我而言，这非常容易，因为我已经编写了自己的Web客户端来处理响应数据。在那里，我用以下代码覆盖了功能 GetWebRequest ：

protected override WebRequest GetWebRequest(Uri _uri)
{

    HttpWebRequest _webRequest = base.GetWebRequest(_uri) as HttpWebRequest;

    if (_webRequest != null)
    {
        // no buffer
        _webRequest.AllowWriteStreamBuffering = false;
        // if data size overhanded in constructur(necessary with AllowWriteStreamBuffering ==  false)
        if (this.ContentLength > 0)
            _webRequest.ContentLength = this.ContentLength;
        // if NO data size overhanded in constructur(necessary with AllowWriteStreamBuffering ==  false)
        else
            _webRequest.SendChunked = true;
        // if timeout overhanded in constructur
        if (this.TimeOut > 0)
            _webRequest.Timeout = this.TimeOut * 1000;

        return (_webRequest);
    }
    else
        return (null);
}

如何使用WebClient将大数据上传到主机

2 个答案: