使用NLTK自定义POS标记(错误)

时间:2016-02-18 07:49:19

标签: python nlp nltk

我正在尝试将自己的简单自定义标记器与nltk默认标记器结合使用,在本例中为感知器标记器。

我的代码如下(基于this answer):

import nltk.tag, nltk.data

default_tagger = nltk.data.load(nltk.tag._POS_TAGGER)
model = {'example_one': 'VB' 'example_two': 'NN'}
tagger = nltk.tag.UnigramTagger(model=model, backoff=default_tagger)

然而,这会产生以下错误:

  File "nltk_test.py", line 24, in <module>
    default_tagger = nltk.data.load(nltk.tag._POS_TAGGER)
  AttributeError: 'module' object has no attribute '_POS_TAGGER'

我尝试通过将默认标记符更改为:

来解决此问题
from nltk.tag.perceptron import PerceptronTagger
default_tagger = PerceptronTagger()

但后来我收到以下错误:

  File "nltk_test.py", line 26, in <module>
    tagger = nltk.tag.UnigramTagger(model=model, backoff=default_tagger)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nltk/tag/sequential.py", line 340, in __init__
    backoff, cutoff, verbose)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nltk/tag/sequential.py", line 284, in __init__
    ContextTagger.__init__(self, model, backoff)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nltk/tag/sequential.py", line 125, in __init__
    SequentialBackoffTagger.__init__(self, backoff)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nltk/tag/sequential.py", line 50, in __init__
    self._taggers = [self] + backoff._taggers
AttributeError: 'PerceptronTagger' object has no attribute '_taggers'

通过nltk.tag documentation查看_POS_TAGGER似乎不再存在。但是,将其更改为_pos_tagpos_tag也无效。

2 个答案:

答案 0 :(得分:4)

快速回答:现在使用nltk 3.0.1 pip install nltk==3.0.1

更好的答案:他们去年9月更改了树库标记器并且还有很多其他后果(我们目前固定在3.0.1上,因为新标记器至少对我们的需求更差)。

这似乎有效,但我不确定代码是否正确:

class BackoffTagger:
    def __init__(self):
        self._taggers = [PerceptronTagger()]

model = {'example_one': 'VB', 'example_two': 'NN'}
tagger = nltk.tag.UnigramTagger(model=model, backoff=BackoffTagger())
tagger.tag(['example_one'])
>>> [('example_one', 'VB')]

答案 1 :(得分:0)

尝试以下自定义标记:

import nltk.tag, nltk.data
from nltk.tag.perceptron import PerceptronTagger
default_tagger = PerceptronTagger()

使用自定义标签定义模型:

model={"paining": "Reaction", "Itching":"Reaction", "Removed":"Reaction", "skin":"site"}

class BackoffTagger:
    def __init__(self):
        self._taggers = [PerceptronTagger()]

tagger = nltk.tag.UnigramTagger(model=model, backoff=BackoffTagger())
tagger.tag(['skin'])

输出

  

[('skin','site')]