读取XML文件以写入SQL数据库时出现System.OutOfMemoryException

时间:2018-03-22 17:15:38

标签: c# .net out-of-memory sqlxml sqlclient

目前,我正在读取XML文件(我的程序以前在文件系统中创建),并使用SQLXml对象将该XML文件的内容写入数据库。

我们每年只会遇到一次年度数据处理问题。每月运行相同的应用程序时,但由于文件大小较小,因此不会发生此错误。异常似乎是由XML文件的大小引起的。最初,我们有一个接近100MB的文件导致此异常非常一致,并更新了代码,以便它是第一个处理的文件,这似乎缓解了这个问题;但是,如果该文件超过93MB,我们又开始收到错误。

这次应用程序出错了,是在处理了63个单独的文件之后,估计总计大约54MB,然后尝试处理大约53MB的单个文件。当我们重新启动应用程序时,它将首先处理失败的文件。

private SqlXml RetrieveXmlForDatabase(string filePath)
{
    using (var reader = new StreamReader(filePath))
    {
        using (var xmlReader = new XmlTextReader(reader))
        {
            return new SqlXml(xmlReader);
        }
    }
}

public void WriteFormDataXMLToDatabase(int ID, string xmlFilePath)
{
    List<SqlParameter> insertParameters = new List<SqlParameter>();

    SqlXml formDataXml = RetrieveXmlForDatabase(xmlFilePath);

    insertParameters.Add(new SqlParameter("@ID", SqlDbType.Int) { Value = ID});
    insertParameters.Add(new SqlParameter("@FormData", SqlDbType.Xml) { Value = formDataXml });

    ExecuteNonQuery("[spname]", insertParameters.ToArray());
}

我收到的System.OutOfMemoryException发生在RetrieveXmlForDatabase方法中。

========================================================================
The following Exception ocurred during application execution:
========================================================================
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.IO.MemoryStream.set_Capacity(Int32 value)
   at System.IO.MemoryStream.EnsureCapacity(Int32 value)
   at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.Xml.XmlEncodedRawTextWriter.EncodeChars(Int32 startOffset, Int32 endOffset, Boolean writeAllToStream)
   at System.Xml.XmlEncodedRawTextWriter.FlushBuffer()
   at System.Xml.XmlEncodedRawTextWriter.RawText(Char* pSrcBegin, Char* pSrcEnd)
   at System.Xml.XmlEncodedRawTextWriter.RawText(String s)
   at System.Xml.XmlEncodedRawTextWriter.WriteFullEndElement(String prefix, String localName, String ns)
   at System.Xml.XmlWellFormedWriter.WriteFullEndElement()
   at System.Xml.XmlWriter.WriteNode(XmlReader reader, Boolean defattr)
   at System.Data.SqlTypes.SqlXml.CreateMemoryStreamFromXmlReader(XmlReader reader)
   at System.Data.SqlTypes.SqlXml..ctor(XmlReader value)
   at System.Data.SqlClient.SqlParameter.CoerceValue(Object value, MetaType destinationType, Boolean& coercedToDataFeed, Boolean& typeChanged, Boolean allowStreaming)
   at System.Data.SqlClient.SqlParameter.GetCoercedValue()
   at System.Data.SqlClient.SqlParameter.Validate(Int32 index, Boolean isCommandProc)
   at System.Data.SqlClient.SqlCommand.SetUpRPCParameters(_SqlRPC rpc, Int32 startCount, Boolean inSchema, SqlParameterCollection parameters)
   at System.Data.SqlClient.SqlCommand.BuildRPC(Boolean inSchema, SqlParameterCollection parameters, _SqlRPC& rpc)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
========================================================================

当前的解决方法是让应用程序在此实例中失败,然后重新执行应用程序。虽然这是可以的,因为我们有检查以防止数据丢失,但如果错误没有首先发生,那将是首选。

有没有人对如何防止这种情况有任何想法。

0 个答案:

没有答案