在xml文件中存在大注释时,如何提高xerces解析器的性能?

时间:2015-02-12 14:18:10

标签: java xml performance xerces

我正在使用以下代码使用xerces 2.11解析xml文件:

@Test
public void testXercesPerformance() throws IOException, SAXException, ParserConfigurationException
{
    final SAXParserFactory spf = SAXParserFactory.newInstance();
    final SAXParser parser = spf.newSAXParser();
    final XMLReader xmlReader = parser.getXMLReader();
    final InputSource inputSource = new InputSource(new BufferedInputStream(new FileInputStream(new File("./some.xml")), 8192));
    xmlReader.parse(inputSource);
}

然而,当xml文件在开头只包含几个xml元素并且在结尾包含大注释(总文件大小约为10MB)时,性能非常差。在解析过程中,解析器连续分配新的字符串,最终总计1.3TB的已分配字符串(并非所有字符串同时分配)。解析本身需要4分钟才能完成。

我用于测试的文件始于:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <version>1.0-SNAPSHOT</version>
    <artifactId>helloworld-secure</artifactId>
    <dependencies>
        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-servlet</artifactId>
            <version>7.4.5.v20110725</version>
        </dependency>
        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-security</artifactId>
            <version>7.4.5.v20110725</version>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
            <version>2.5</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>appassembler-maven-plugin</artifactId>
                <version>1.1.1</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals><goal>assemble</goal></goals>
                        <configuration>
                            <assembleDirectory>target</assembleDirectory>
                            <programs>
                                <program>
                                    <mainClass>HelloWorld</mainClass>
                                    <name>webapp</name>
                                </program>
                            </programs>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
            <source>1.6</source>
                    <target>1.6</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>
<!-- 
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <version>1.0-SNAPSHOT</version>
    <artifactId>helloworld-secure</artifactId>
    <dependencies>

然后,它会从未注释的部分重复数百次依赖,直到达到近10MB的大小,并以:

结束。
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>appassembler-maven-plugin</artifactId>
                <version>1.1.1</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals><goal>assemble</goal></goals>
                        <configuration>
                            <assembleDirectory>target</assembleDirectory>
                            <programs>
                                <program>
                                    <mainClass>HelloWorld</mainClass>
                                    <name>webapp</name>
                                </program>
                            </programs>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
            <source>1.6</source>
                    <target>1.6</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>
-->

这种性能不佳的原因是什么?如何配置解析器以提高性能?

2 个答案:

答案 0 :(得分:1)

以前(好的,超过10年前)的问题报告为XERCESJ-970。自2013年中期以来,它已在revision 1507079 of xerces-j trunk中修复。

问题是XMLStringBuffer内的线性增长缓冲区通常需要重新分配。

我的案例中的修复是使用r1507079中的补丁重建xerces 2.11。

答案 1 :(得分:0)

您可以使用StreamFilter类提供的EventFilterXmlInputFactory,这两个类允许您在实际读者解析之前拦截解析行为。 StreamCommentFilter是一个类,它将阻止任何注释被解析。我使用了您的示例并制作了一个20mb的文件,并在我的计算机上启用或禁用过滤器时快速解析它。我的电脑碰巧很快但是在较慢的电脑上可能会有所不同。

为方便起见进口:

import javax.xml.stream.StreamFilter;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamReader;
import javax.xml.stream.events.XMLEvent;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMResult;
import javax.xml.transform.stax.StAXSource;
import javax.xml.transform.stream.StreamSource;

import org.w3c.dom.Document;

// Create our factory and make sure its namespace aware.
XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();
xmlInputFactory.setProperty(XMLInputFactory.IS_NAMESPACE_AWARE, true);
// create the filtered reader that will not allow any comments to be parsed
XMLStreamReader reader = xmlInputFactory.createFilteredReader(
        xmlInputFactory.createXMLStreamReader(new StreamSource(new File("./some.xml"))), 
        new StreamCommentRemovalFilter());

// transform our XmlStreamReader into a Document using a Transformer
TransformerFactory transFactory = TransformerFactory.newInstance();
Transformer transformer = transFactory.newTransformer();

DOMResult result = new DOMResult();
transformer.transform(new StAXSource(reader), result);
Document document = (Document) result.getNode();
// do something with your document

StreamFilter实现,这将不允许在解析过程中将任何注释实际放入Java对象中。

public static class StreamCommentRemovalFilter implements StreamFilter {
    @Override
    public boolean accept(XMLStreamReader reader) {
        // if its a comment dont parse it
        if(reader.getEventType() == XMLEvent.COMMENT) {
            return false;
        }
        return true;
    }
}

在您的示例中,您还包括命名空间和Schema,因此我假设您要进行一些验证,如果是这样,您仍然可以使用DOMSource类来执行它,并使用解析后的Document上面的代码。

final SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
final Source schemaSource = new StreamSource(new URL("http://maven.apache.org/xsd/maven-4.0.0.xsd").openStream());
final Schema schema = schemaFactory.newSchema(schemaSource);

schema.newValidator().validate(new DOMSource(document.getFirstChild()));