按字节大小预测拆分XML文件

时间:2017-04-10 15:54:16

标签: c# xml linq linq-to-xml

我有XML消息xmlStr,必须将其拆分为小于或等于maxSizeBytes的较小XML消息。这是通过获取文档的根目录,并将第一个子文件作为较小XML的基础,并获取一些<Smt>元素,并将它们的数量放入新形成的(较小的)XML消息中。

<?xml version="1.0"?>
<Bas>
  <Hdr>
    <Smt>...</Smt>
    <Smt>...</Smt>
    <Smt>...</Smt>
   </Hdr>
</Bas>

目前,我正在测量整个邮件大小int smtNodesPerMessage = (int)Math.Ceiling((double)ASCIIEncoding.ASCII.GetByteCount(xmlStr) / (double)maxSizeBytes);,然后是 将smtNodesPerMessage个节点放入较小的XML中:

//doc is original XDocument message
XDocument splitXML = new XDocument(new XElement(doc.Root.Name,                                              
                                   doc.Root.Descendants("Hdr")));
splitXML.Root.Add(batchOfSmt);

我很快发现,较小的XML文件的字节大小大于maxSizeBytes,因为XDocument为每条消息添加了额外的字符,增加了字节大小。

1 个答案:

答案 0 :(得分:2)

基本算法是:

  • 获取具有空Hdr元素的文档的大小。请注意,默认编码为UTF-8。所以我使用Encoding.Default.GetByteCount来计算文档的大小及其元素。
  • 为每个子文档克隆此空-hdr文档
  • 对于eash Smt元素,在添加之前检查子文档大小是否超过最大值

带注释的代码

var doc = XDocument.Load("data.xml");
var hdr = xdoc.Root.Element("Hdr");
var elements = hdr.Elements().ToList(); 
hdr.RemoveAll(); // we can remove child elements, because they are stored in a list
hdr.Value = "";  // otherwise xdoc will compact empty element to <Hdr/>

// calculating size of sub-document 'template'
var sb = new StringBuilder();
using (XmlWriter writer = XmlWriter.Create(sb))
    doc.Save(writer);
var outerSizeInBytes = Encoding.Default.GetByteCount(sb.ToString());

var maxSizeInBytes = 100;
var subDocumentIndex = 0; // used just for naming sub-document files
var subDocumentSizeBytes = outerSizeInBytes; // initial size of any sub-document
var subDocument = new XDocument(doc); // clone 'template'

foreach (var smt in elements)
{
    var currentElementSizeBytes = Encoding.Default.GetByteCount(smt.ToString());

    if (maxSizeInBytes < subDocumentSizeBytes + currentElementSizeBytes
        && subDocumentSizeBytes != outerSizeInBytes) // case when first element is too big
    {
        subDocument.Save($"doc{++subDocumentIndex}.xml");
        subDocument = new XDocument(doc);
        subDocumentSizeBytes = outerSizeInBytes;
    }

    subDocument.Root.Element("Hdr").Add(smt);
    subDocumentSizeBytes += currentElementSizeBytes;
}

// if current sub-document has elements added, save it too
if (outerSizeInBytes < subDocumentSizeBytes)
    subDocument.Save($"doc{++subDocumentIndex}.xml");

当source和max size为250字节时,您将获得三个文档

<?xml version="1.0"?>
<Bas>
  <Hdr>
    <Smt>Lorem Ipsum is simply dummy text of the printing and typesetting industry.</Smt>
    <Smt>Contrary to popular belief, Lorem Ipsum is not simply random text.</Smt>
    <Smt>It has survived not only five centuries, 
 but also the leap into electronic typesetting, remaining essentially unchanged.</Smt>
    <Smt>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</Smt>
  </Hdr>
</Bas>

doc1(223字节):

<?xml version="1.0" encoding="utf-8"?>
<Bas>
  <Hdr>
    <Smt>Lorem Ipsum is simply dummy text of the printing and typesetting industry.</Smt>
    <Smt>Contrary to popular belief, Lorem Ipsum is not simply random text.</Smt>
  </Hdr>
</Bas>

doc2(259字节,单个元素):

<?xml version="1.0" encoding="utf-8"?>
<Bas>
  <Hdr>
    <Smt>It has survived not only five centuries, 
 but also the leap into electronic typesetting, remaining essentially unchanged.</Smt>
  </Hdr>
</Bas>

doc3(128字节,最后一个)

<?xml version="1.0" encoding="utf-8"?>
<Bas>
  <Hdr>
    <Smt>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</Smt>
  </Hdr>
</Bas>