答案 0 :(得分：0)
不幸的是，OP作为代表示例提供的文件没有被标记。因此，没有直接信息指示给定的文本片段是属于标题，摘要，参考文献还是属于哪个部分。因此，没有确定的方法来识别这些部分，而仅仅是启发式算法，也就是受过教育的猜测，具有或多或少的错误率。
对于OP提供的样本文件，可以通过简单检查每行的第一个字母的字体来实现部件的识别。
以下类构成了一个简单的框架，用于提取语义文本部分，这些部分可以通过它们各自的特征来识别，以及通过仅检查字体来识别OP示例文件中的部分的示例每行的第一个字符。
由于我还没有使用Java版本的PDFBox，OP宣称Java解决方案也可以，所以框架是用Java实现的。它基于PDFBox的当前开发版本2.1.0-SNAPSHOT。
PDFTextSectionStripper 该类构成框架的中心。它源自PDFBox PdfTextStripper，并通过识别由TextSectionDefinition实例列表配置的文本部分来扩展该类，请参见下文。调用PdfTextStripper方法getText后，已识别的部分将作为TextSection个实例的列表提供，请参阅下文。
public class PDFTextSectionStripper extends PDFTextStripper
{
//
// constructor
//
public PDFTextSectionStripper(List<TextSectionDefinition> sectionDefinitions) throws IOException
{
super();
this.sectionDefinitions = sectionDefinitions;
}
//
// Section retrieval
//
/**
* @return an unmodifiable list of text sections recognized during {@link #getText(PDDocument)}.
*/
public List<TextSection> getSections()
{
return Collections.unmodifiableList(sections);
}
//
// PDFTextStripper overrides
//
@Override
protected void writeLineSeparator() throws IOException
{
super.writeLineSeparator();
if (!currentLine.isEmpty())
{
boolean matched = false;
if (!(currentHeader.isEmpty() && currentBody.isEmpty()))
{
TextSectionDefinition definition = sectionDefinitions.get(currentSectionDefinition);
switch (definition.multiLine)
{
case multiLine:
if (definition.matchPredicate.test(currentLine))
{
currentBody.add(new ArrayList<>(currentLine));
matched = true;
}
break;
case multiLineHeader:
case multiLineIntro:
boolean followUpMatch = false;
for (int i = definition.multiple ? currentSectionDefinition : currentSectionDefinition + 1;
i < sectionDefinitions.size(); i++)
{
TextSectionDefinition followUpDefinition = sectionDefinitions.get(i);
if (followUpDefinition.matchPredicate.test(currentLine))
{
followUpMatch = true;
break;
}
}
if (!followUpMatch)
{
currentBody.add(new ArrayList<>(currentLine));
matched = true;
}
break;
case singleLine:
System.out.println("Internal error: There can be no current header or body as long as the current definition is single line only");
}
if (!matched)
{
sections.add(new TextSection(definition, currentHeader, currentBody));
currentHeader.clear();
currentBody.clear();
if (!definition.multiple)
currentSectionDefinition++;
}
}
if (!matched)
{
while (currentSectionDefinition < sectionDefinitions.size())
{
TextSectionDefinition definition = sectionDefinitions.get(currentSectionDefinition);
if (definition.matchPredicate.test(currentLine))
{
matched = true;
switch (definition.multiLine)
{
case singleLine:
sections.add(new TextSection(definition, currentLine, Collections.emptyList()));
if (!definition.multiple)
currentSectionDefinition++;
break;
case multiLineHeader:
currentHeader.addAll(new ArrayList<>(currentLine));
break;
case multiLine:
case multiLineIntro:
currentBody.add(new ArrayList<>(currentLine));
break;
}
break;
}
currentSectionDefinition++;
}
}
if (!matched)
{
System.out.println("Could not match line.");
}
}
currentLine.clear();
}
@Override
protected void endDocument(PDDocument document) throws IOException
{
super.endDocument(document);
if (!(currentHeader.isEmpty() && currentBody.isEmpty()))
{
TextSectionDefinition definition = sectionDefinitions.get(currentSectionDefinition);
sections.add(new TextSection(definition, currentHeader, currentBody));
currentHeader.clear();
currentBody.clear();
}
}
@Override
protected void writeString(String text, List<TextPosition> textPositions) throws IOException
{
super.writeString(text, textPositions);
currentLine.add(textPositions);
}
//
// member variables
//
final List<TextSectionDefinition> sectionDefinitions;
int currentSectionDefinition = 0;
final List<TextSection> sections = new ArrayList<>();
final List<List<TextPosition>> currentLine = new ArrayList<>();
final List<List<TextPosition>> currentHeader = new ArrayList<>();
final List<List<List<TextPosition>>> currentBody = new ArrayList<>();
}
TextSectionDefinition 此类指定文本节类型的属性，名称，匹配谓词，MultiLine属性和多次出现标记。
名称纯粹是描述性的。
匹配谓词是一个函数，它提供有关文本行上字符的详细信息，并返回此行是否与相关文本部分类型匹配。
MultiLine属性可以采用四种不同的值之一：
singleLine - 仅适用于由一行组成的部分; multiLine - 用于多行部分，其中每行必须与谓词匹配; multiLineHeader - 对于多行部分，其中第一行只需要匹配谓词，第一行是标题行; multiLineIntro - 对于多行部分，其中第一行只需要与谓词匹配，第一行是该部分的常规部分，可能只是由一个特殊的标记词引入。多次出现标志表示是否可以存在此类文本部分的多个实例。
public class TextSectionDefinition
{
public enum MultiLine
{
singleLine, // A single line without text body, e.g. title
multiLine, // Multiple lines, all match predicate, e.g. emails
multiLineHeader, // Multiple lines, first line matches as header, e.g. h1
multiLineIntro // Multiple lines, first line matches inline, e.g. abstract
}
public TextSectionDefinition(String name, Predicate<List<List<TextPosition>>> matchPredicate, MultiLine multiLine, boolean multiple)
{
this.name = name;
this.matchPredicate = matchPredicate;
this.multiLine = multiLine;
this.multiple = multiple;
}
final String name;
final Predicate<List<List<TextPosition>>> matchPredicate;
final MultiLine multiLine;
final boolean multiple;
}
TextSection 此类表示此框架识别的文本部分。
public class TextSection
{
public TextSection(TextSectionDefinition definition, List<List<TextPosition>> header, List<List<List<TextPosition>>> body)
{
this.definition = definition;
this.header = new ArrayList<>(header);
this.body = new ArrayList<>(body);
}
@Override
public String toString()
{
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append(definition.name).append(": ");
if (!header.isEmpty())
stringBuilder.append(toString(header));
stringBuilder.append('\n');
for (List<List<TextPosition>> bodyLine : body)
{
stringBuilder.append(" ").append(toString(bodyLine)).append('\n');
}
return stringBuilder.toString();
}
String toString(List<List<TextPosition>> words)
{
StringBuilder stringBuilder = new StringBuilder();
boolean first = true;
for (List<TextPosition> word : words)
{
if (first)
first = false;
else
stringBuilder.append(' ');
for (TextPosition textPosition : word)
{
stringBuilder.append(textPosition.getUnicode());
}
}
// cf. https://stackoverflow.com/a/7171932/1729265
return Normalizer.normalize(stringBuilder, Form.NFKC);
}
final TextSectionDefinition definition;
final List<List<TextPosition>> header;
final List<List<List<TextPosition>>> body;
}
关于Normalizer.normalize(stringBuilder, Form.NFKC)来电cf. this answer到堆栈溢出问题"Separating Unicode ligature characters"。
On可以使用这个框架和非常简单的匹配谓词来识别OP提供的代表性样本中的部分：
List<TextSectionDefinition> sectionDefinitions = Arrays.asList(
new TextSectionDefinition("Titel", x->x.get(0).get(0).getFont().getName().contains("CMBX12"), MultiLine.singleLine, false),
new TextSectionDefinition("Authors", x->x.get(0).get(0).getFont().getName().contains("CMR10"), MultiLine.multiLine, false),
new TextSectionDefinition("Institutions", x->x.get(0).get(0).getFont().getName().contains("CMR9"), MultiLine.multiLine, false),
new TextSectionDefinition("Addresses", x->x.get(0).get(0).getFont().getName().contains("CMTT9"), MultiLine.multiLine, false),
new TextSectionDefinition("Abstract", x->x.get(0).get(0).getFont().getName().contains("CMBX9"), MultiLine.multiLineIntro, false),
new TextSectionDefinition("Section", x->x.get(0).get(0).getFont().getName().contains("CMBX12"), MultiLine.multiLineHeader, true)
);
PDDocument document = PDDocument.load(resource);
PDFTextSectionStripper stripper = new PDFTextSectionStripper(sectionDefinitions);
stripper.getText(document);
System.out.println("Sections:");
List<String> texts = new ArrayList<>();
for (TextSection textSection : stripper.getSections())
{
String text = textSection.toString();
System.out.println(text);
texts.add(text);
}
Files.write(new File(RESULT_FOLDER, "Wang05a.txt").toPath(), texts);
（ExtractTextSections.java测试方法testWang05a）
缩短的结果：
Titel: How to Break MD5 and Other Hash Functions
Authors:
Xiaoyun Wang and Hongbo Yu
Institutions:
Shandong University, Jinan 250100, China,
Addresses:
xywang@sdu.edu.cn, yhb@mail.sdu.edu.cn
Abstract:
Abstract. MD5 is one of the most widely used cryptographic hash func-
tions nowadays. It was designed in 1992 as an improvement of MD4, and
...
Section: 1 Introduction
People know that digital signatures are very important in information security.
The security of digital signatures depends on the cryptographic strength of the
...
Section: 2 Description of MD5
In order to conveniently describe the general structure of MD5, we first recall
the iteration process for hash functions.
...
Section: 3 Differential Attack for Hash Functions
3.1 The Modular Differential and the XOR Differential
The most important analysis method for hash functions is differential attack
...
Section: 4 Differential Attack on MD5
4.1 Notation
Before presenting our attack, we first introduce some notation to simplify the
...
Section: 5 Summary
In this paper we described a powerful attack against hash functions, and in
particular showed that finding a collision of MD5 is easily feasible.
...
Section: Acknowledgements
It is a pleasure to acknowledge Dengguo Feng for the conversations that led to
this research on MD5. We would like to thank Eli Biham, Andrew C. Yao, and
...
Section: References
1. E. Biham, A. Shamir. Differential Cryptanalysis of the Data Encryption Standard,
Springer-Verlag, 1993.
...
对于更通用的文本部分识别，显然不能指望这些特定的TeX字体用于发信号通知特定的文本部分。相反，可能必须查看字体大小（记住不要采用简单的字体大小属性，而是根据转换和文本矩阵进行缩放！），对齐等。可能需要首先扫描文档以确定常见的文本大小等
如果在同一杂志中发布了多个文档，识别谓词可能实际上就像上面的例子一样简单，因为在这种情况下，作者通常必须坚持非常具体的布局和格式规则。