我正在使用以下代码使用xerces 2.11解析xml文件:
@Test
public void testXercesPerformance() throws IOException, SAXException, ParserConfigurationException
{
final SAXParserFactory spf = SAXParserFactory.newInstance();
final SAXParser parser = spf.newSAXParser();
final XMLReader xmlReader = parser.getXMLReader();
final InputSource inputSource = new InputSource(new BufferedInputStream(new FileInputStream(new File("./some.xml")), 8192));
xmlReader.parse(inputSource);
}
然而,当xml文件在开头只包含几个xml元素并且在结尾包含大注释(总文件大小约为10MB)时,性能非常差。在解析过程中,解析器连续分配新的字符串,最终总计1.3TB的已分配字符串(并非所有字符串同时分配)。解析本身需要4分钟才能完成。
我用于测试的文件始于:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<version>1.0-SNAPSHOT</version>
<artifactId>helloworld-secure</artifactId>
<dependencies>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlet</artifactId>
<version>7.4.5.v20110725</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-security</artifactId>
<version>7.4.5.v20110725</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<version>1.1.1</version>
<executions>
<execution>
<phase>package</phase>
<goals><goal>assemble</goal></goals>
<configuration>
<assembleDirectory>target</assembleDirectory>
<programs>
<program>
<mainClass>HelloWorld</mainClass>
<name>webapp</name>
</program>
</programs>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
<!--
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<version>1.0-SNAPSHOT</version>
<artifactId>helloworld-secure</artifactId>
<dependencies>
然后,它会从未注释的部分重复数百次依赖,直到达到近10MB的大小,并以:
结束。 </dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<version>1.1.1</version>
<executions>
<execution>
<phase>package</phase>
<goals><goal>assemble</goal></goals>
<configuration>
<assembleDirectory>target</assembleDirectory>
<programs>
<program>
<mainClass>HelloWorld</mainClass>
<name>webapp</name>
</program>
</programs>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
-->
这种性能不佳的原因是什么?如何配置解析器以提高性能?
答案 0 :(得分:1)
以前(好的,超过10年前)的问题报告为XERCESJ-970。自2013年中期以来,它已在revision 1507079 of xerces-j trunk中修复。
问题是XMLStringBuffer
内的线性增长缓冲区通常需要重新分配。
我的案例中的修复是使用r1507079中的补丁重建xerces 2.11。
答案 1 :(得分:0)
您可以使用StreamFilter
类提供的EventFilter
或XmlInputFactory
,这两个类允许您在实际读者解析之前拦截解析行为。 StreamCommentFilter
是一个类,它将阻止任何注释被解析。我使用了您的示例并制作了一个20mb的文件,并在我的计算机上启用或禁用过滤器时快速解析它。我的电脑碰巧很快但是在较慢的电脑上可能会有所不同。
为方便起见进口:
import javax.xml.stream.StreamFilter;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamReader;
import javax.xml.stream.events.XMLEvent;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMResult;
import javax.xml.transform.stax.StAXSource;
import javax.xml.transform.stream.StreamSource;
import org.w3c.dom.Document;
// Create our factory and make sure its namespace aware.
XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();
xmlInputFactory.setProperty(XMLInputFactory.IS_NAMESPACE_AWARE, true);
// create the filtered reader that will not allow any comments to be parsed
XMLStreamReader reader = xmlInputFactory.createFilteredReader(
xmlInputFactory.createXMLStreamReader(new StreamSource(new File("./some.xml"))),
new StreamCommentRemovalFilter());
// transform our XmlStreamReader into a Document using a Transformer
TransformerFactory transFactory = TransformerFactory.newInstance();
Transformer transformer = transFactory.newTransformer();
DOMResult result = new DOMResult();
transformer.transform(new StAXSource(reader), result);
Document document = (Document) result.getNode();
// do something with your document
StreamFilter
实现,这将不允许在解析过程中将任何注释实际放入Java对象中。
public static class StreamCommentRemovalFilter implements StreamFilter {
@Override
public boolean accept(XMLStreamReader reader) {
// if its a comment dont parse it
if(reader.getEventType() == XMLEvent.COMMENT) {
return false;
}
return true;
}
}
在您的示例中,您还包括命名空间和Schema,因此我假设您要进行一些验证,如果是这样,您仍然可以使用DOMSource
类来执行它,并使用解析后的Document
上面的代码。
final SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
final Source schemaSource = new StreamSource(new URL("http://maven.apache.org/xsd/maven-4.0.0.xsd").openStream());
final Schema schema = schemaFactory.newSchema(schemaSource);
schema.newValidator().validate(new DOMSource(document.getFirstChild()));