Question

我无法通过API大量获取XML数据。有没有人在ASP.net应用程序中获取XML数据更好的解决方案？

也许问题可能是垃圾收集内存不确定，但早些时候它不适用于任何用户，因此我在每次调用后添加了手动内存释放代码并关闭Web响应连接。

欢迎任何改进获取大量数据的建议吗？

   public static XmlDocument LoadXmlFromUrl(string url, bool keepAlive = true, bool throwException = false, int timeout = 15000)
    {
        Log(url, "*** start ***" + url);

        ServicePointManager.DefaultConnectionLimit = 2000;
        string xmlAsText = null;
        HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
        req.KeepAlive = keepAlive;
        req.Timeout = timeout; // 30,000ms = 30 sec default
        try
        {
            bool retried = false;
        retry:
            try
            {
                using (HttpWebResponse webResponse = (HttpWebResponse)req.GetResponse())
                {
                    using (StreamReader responseStream = new StreamReader(webResponse.GetResponseStream(), Encoding.UTF8))
                    {
                        xmlAsText = responseStream.ReadToEnd();
                        responseStream.Close();
                    }
                    webResponse.Close();

                    Log(url, "stream reader (responseStream.ReadToEnd) completed to string.");
                }
            }
            catch (WebException webEx)
            {
                if (!retried)
                {
                    if (webEx.Status == WebExceptionStatus.Timeout || webEx.Status == WebExceptionStatus.KeepAliveFailure)
                    {
                        Log(url, "About to retry for url (6 sec wait): " + url);
                        System.Threading.Thread.Sleep(6000);
                        retried = true;
                        goto retry;
                    }
                }
                else
                {
                    throw webEx;
                }
            }
        }
        catch (Exception ex)
        {
            Log(url, "exception while loading responseStream.ReadToEnd. " + ex.ToString());
        }
        XmlDocument xdoc = null;
        if (!string.IsNullOrEmpty(xmlAsText))
        {
            try
            {
                xdoc = new XmlDocument();
                xdoc.LoadXml(xmlAsText);
                Log(url, "string added to xml (xdoc.LoadXml) completed.");
            }
            catch (Exception ex)
            {
                Log(url, "exception while loading xdoc.LoadXml. " + ex.ToString());
            }
        }
        return xdoc;
    }

例外1：

5/01/2017 12:27:01 p.m. exception while loading responseStream.ReadToEnd.     System.Net.WebException: The operation has timed out**

例外2：

6/01/2017 8:35:59 a.m. exception while loading responseStream.ReadToEnd. System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.

Answer 1

如果

，可能会发生

异常＃1

请求很大。在这种情况下，您可以尝试将C:\Project\Tools\liquibase-3.5.3-bin>liquibase --driver=oracle.jdbc.driver.Oracl eDriver --changeLogFile=C:\Project\Tools\Migrations\liquibase\master.xml --url=" jdbc:oracle:thin:@database:1521:name" --username=user --password=dpass migrate Unexpected error running Liquibase: ORA-00955: name is already used by an existi ng object [Failed SQL: create table int.person ( id int not null primary key, firstname varchar(80), lastname varchar(80) not null, state varchar(2) )]的值增加到相对较大的值，例如3600000（1小时）。
标头HttpWebRequest.ReadWriteTimeout的值与实际响应大小不匹配，服务器不会强制关闭连接。

要追查问题，我建议您设置跟踪。您可以关注https://msdn.microsoft.com/en-us/library/ty48b824(v=vs.110).aspx或在StackOverflow上找到类似的内容。

异常＃2 肯定是因为响应不适合为应用程序分配的内存或其大小超过2Gb而发生。（这一定是非常大的回应）。可能的解决方案是将流缓冲到文件或在运行时处理它（如果可能）。例如：

ContentLength

API调用因超出大量数据而超时

1 个答案: