Question

我收集的名词短语大约有10,000个单词。我想检查这些NP集合的每个新输入文本数据，并提取包含任何这些NP的句子。我不想为每个单词运行循环，因为它使我的代码死得很慢。我正在使用Java和Stanford CoreNLP。

Answer 1

快速简便的方法是使用RegexNER识别字典中所有内容的所有示例，然后检查句子中的非“O”NER标记。

package edu.stanford.nlp.examples;

import edu.stanford.nlp.ling.*;
import edu.stanford.nlp.pipeline.*;
import edu.stanford.nlp.util.*;
import java.util.*;
import java.util.stream.Collectors;

public class FindSentencesWithPhrase {

  public static boolean checkForNamedEntity(CoreMap sentence) {
    for (CoreLabel token : sentence.get(CoreAnnotations.TokensAnnotation.class)) {
      if (token.ner() != null && !token.ner().equals("O")) {
        return true;
      }
    }
    return false;
  }

  public static void main(String[] args) {
    Properties props = new Properties();
    props.setProperty("annotators", "tokenize,ssplit,pos,lemma,regexner");
    props.setProperty("regexner.mapping", "phrases.rules");
    StanfordCoreNLP pipeline = new StanfordCoreNLP(props);
    String exampleText = "This sentence contains the phrase \"ice cream\"." +
        "This sentence is not of interest.  This sentences contains pizza.";
    Annotation ann = new Annotation(exampleText);
    pipeline.annotate(ann);
    for (CoreMap sentence : ann.get(CoreAnnotations.SentencesAnnotation.class)) {
      if (checkForNamedEntity(sentence)) {
        System.out.println("---");
        System.out.println(sentence.get(CoreAnnotations.TokensAnnotation.class).
            stream().map(token -> token.word()).collect(Collectors.joining(" ")));
      }
    }
  }
}

文件“phrase.rules”应如下所示：

ice cream       PHRASE_OF_INTEREST      MISC    1
pizza   PHRASE_OF_INTEREST      MISC    1

检查单词集合中输入文本中的单词

1 个答案: