如何将非常大的文件存储到数据库记录中

时间:2019-04-16 21:46:07

标签: c# asp.net sql-server xml stored-procedures

客户端使用我们的Web应用程序来解析数据库的XML文件。然后将来自数据库的XML文件中的信息用于其他各种用途。我现在是流读取和流写入文件。当文件较小时,它们通常被解析为字符串,然后插入数据库记录中。问题在于文件太大,计算机没有足够的RAM来处理那么大的字符串,如何处理此文件并将其仍存储在数据库中?

using (var reader = new StreamReader(outputPath))
                {
                    string line = string.Empty;
                    bool save = false;
                    using (var sWriter = new StreamWriter(inputPath))
                    {
                        while ((line = reader.ReadLine()) != null)
                        {
                            System.Diagnostics.Debug.WriteLine(line);
                            if (line.Contains("<SaveDataFromScout>"))
                            {
                                sWriter.WriteLine("<SaveDataFromScout>");
                                save = true;
                                continue;
                            }
                            else if (line.Contains("</SaveDataFromScout>"))
                            {
                                sWriter.WriteLine(line);
                                save = false;
                            }

                            if (save)
                            {
                                if (line.Contains("ELEMENT TEXT"))
                                    line = line.Replace("ELEMENT TEXT", "ELEMENTTEXT");

                                sWriter.WriteLine(line);
                            }


                        }
                    }
                }

                //string workbookXML = System.IO.File.ReadAllText(outputPath);

                transactionID = DAC.ExecuteScalar
                    (
                        db.ConnectionString,
                        "dbo.cwi_InsertTransaction",
                        new SqlParameter("@TransactionTypeID", transactionTypeID),
                        new SqlParameter("@UploadedFileName", fileDataLink),
                        //new SqlParameter("@UploadedFileXml", workbookXML),
                        new SqlParameter("@CurrentUserID", CurrentUser.UserID)
                    );

您是否看到注释掉的代码,该代码以前已被注释掉为字符串,然后解析为数据库?好的,这可以,但是不适用于888MB左右的文件。

1 个答案:

答案 0 :(得分:1)

Load the file directly into SQL Server, XML column type using bulk load:

INSERT INTO T(XmlCol)  
SELECT * FROM OPENROWSET(  
   BULK 'c:\SampleFolder\SampleData3.txt',  
   SINGLE_BLOB) AS x;  

More documentation: https://docs.microsoft.com/en-us/sql/relational-databases/import-export/examples-of-bulk-import-and-export-of-xml-documents-sql-server?view=sql-server-2017