Question

我正在尝试使用Apache Tika解析pdf文件，使用ByteArrayInputStream进行二进制文件...并开始收到某些pdf文件的错误，而且有些解析得很好.. 之前我能够解析同样的pdf文件使用Tika，但现在当我尝试使用ByteArrayInputStream时，我开始收到错误..我认为ByteArray存在一些问题这是我得到的错误..

org.apache.tika.exception.TikaException: Unexpected RuntimeException from org.apache.tika.parser.pdf.PDFParser@652489c0

这是我的代码......

if (page.isBinary()) {
   handleBinary(page, curURL);
}

public int handleBinary(Page page, WebURL curURL) {
    try {
          binaryParser.parse(page.getBinaryData());
          page.setText(binaryParser.getText());
          handleMetaData(page, binaryParser.getMetaData());


          //System.out.println(" pdf url " +page.getWebURL().getURL());
          //System.out.println("Text" +page.getText());
    } catch (Exception e) {
          // TODO: handle exception
    }
          return PROCESS_OK;
}

        public class BinaryParser {

            private String text;
            private Map<String, String> metaData;

            private Tika tika;

            public BinaryParser() {
                tika = new Tika();
            }

            public void parse(byte[] data) {
                InputStream is = null;
                try {
                    is = new ByteArrayInputStream(data);
                    text = null;
                    Metadata md = new Metadata();
                    metaData = new HashMap<String, String>();
                    text = tika.parseToString(is, md).trim();
                    processMetaData(md);
                } catch (Exception e) {
                    e.printStackTrace();
                } finally {
                    IOUtils.closeQuietly(is);
                }
            }

            public String getText() {
                return text;
            }

            public void setText(String text) {
                this.text = text;
            }


            private void processMetaData(Metadata md){
                if ((getMetaData() == null) || (!getMetaData().isEmpty())) {
                    setMetaData(new HashMap<String, String>());
                }
                for (String name : md.names()){
                    getMetaData().put(name.toLowerCase(), md.get(name));
                }
            }

            public Map<String, String> getMetaData() {
                return metaData;
            }

            public void setMetaData(Map<String, String> metaData) {
                this.metaData = metaData;
            }

        }

    public class Page {

        private WebURL url;

        private String html;

        // Data for textual content
        private String text;

        private String title;

        private String keywords;
        private String authors;
        private String description;
        private String contentType;
        private String contentEncoding;

        private byte[] binaryData;

        private List<WebURL> urls;

        private ByteBuffer bBuf;

        private final static String defaultEncoding = Configurations
                .getStringProperty("crawler.default_encoding", "UTF-8");

        public boolean load(final InputStream in, final int totalsize,
                final boolean isBinary) {
            if (totalsize > 0) {
                this.bBuf = ByteBuffer.allocate(totalsize + 1024);
            } else {
                this.bBuf = ByteBuffer.allocate(PageFetcher.MAX_DOWNLOAD_SIZE);
            }
            final byte[] b = new byte[1024];
            int len;
            double finished = 0;
            try {
                while ((len = in.read(b)) != -1) {
                    if (finished + b.length > this.bBuf.capacity()) {
                        break;
                    }
                    this.bBuf.put(b, 0, len);
                    finished += len;
                }
            } catch (final BufferOverflowException boe) {
                System.out.println("Page size exceeds maximum allowed.");
                return false;
            } catch (final Exception e) {
                System.err.println(e.getMessage());
                return false;
            }

            this.bBuf.flip();
            if (isBinary) {
                binaryData = new byte[bBuf.limit()];
                bBuf.get(binaryData);
            } else {
                this.html = "";
                this.html += Charset.forName(defaultEncoding).decode(this.bBuf);
                this.bBuf.clear();
                if (this.html.length() == 0) {
                    return false;
                }
            }
            return true;
        }
    public boolean isBinary() {
        return binaryData != null;
    }

    public byte[] getBinaryData() {
        return binaryData;
    }

任何建议我做错了什么...... !!

更新： - 升级到pdfbox 1.6.0版本后，我开始收到一些pdf的错误...

Parsing Error, Skipping Object
java.io.IOException: expected='endstream' actual='' org.apache.pdfbox.io.PushBackInputStream@70dbdc4b
    at org.apache.pdfbox.pdfparser.BaseParser.parseCOSStream(BaseParser.java:439)
    at org.apache.pdfbox.pdfparser.PDFParser.parseObject(PDFParser.java:552)
    at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:184)
    at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1088)
    at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:1053)

对于某些pdf这个错误...

 Did not found XRef object at specified startxref position 0
Invalid dictionary, found: '' but expected: '/'
 WARN [Crawler 2] Did not found XRef object at specified startxref position 0

Answer 1

这是 PDFBox 版本1.4.0的已知错误。只需更新为 PDFBox 1.5.0 + 。

检查release notes：

[PDFBOX-578] PDPageNode.getCount中的NPE NullPointerException

这是JIRA ticket。

解析二进制文件时出错...（主要是PDF）

1 个答案: