Question

我尝试使用spaCy编写最小的文本分类器。我编写了以下代码片段来训练文本分类程序（没有训练整个NLP管道）：

import spacy
from spacy.pipeline import TextCategorizer
nlp = spacy.load('en')

doc1 = u'This is my first document in the dataset.'
doc2 = u'This is my second document in the dataset.'

gold1 = u'Category1'
gold2 = u'Category2'

textcat = TextCategorizer(nlp.vocab)
textcat.add_label('Category1')
textcat.add_label('Category2')
losses = {}
optimizer = textcat.begin_training()
textcat.update([doc1, doc2], [gold1, gold2], losses=losses, sgd=optimizer)

但是当我运行它时，它会返回一个错误。这是我开始时给我的追溯：

Traceback (most recent call last):
  File "C:\Users\Reuben\Desktop\Classification\Classification\Training.py", line
 16, in <module>
    textcat.update([doc1, doc2], [gold1, gold2], losses=losses, sgd=optimizer)
  File "pipeline.pyx", line 838, in spacy.pipeline.TextCategorizer.update
  File "D:\Program Files\Anaconda2\lib\site-packages\thinc\api.py", line 61, in
begin_update
    X, inc_layer_grad = layer.begin_update(X, drop=drop)
  File "D:\Program Files\Anaconda2\lib\site-packages\thinc\api.py", line 176, in
 begin_update
    values = [fwd(X, *a, **k) for fwd in forward]
  File "D:\Program Files\Anaconda2\lib\site-packages\thinc\api.py", line 258, in
 wrap
    output = func(*args, **kwargs)
  File "D:\Program Files\Anaconda2\lib\site-packages\thinc\api.py", line 61, in
begin_update
    X, inc_layer_grad = layer.begin_update(X, drop=drop)
  File "D:\Program Files\Anaconda2\lib\site-packages\spacy\_ml.py", line 95, in
_preprocess_doc
    keys = [doc.to_array(LOWER) for doc in docs]
AttributeError: 'unicode' object has no attribute 'to_array'

我该如何解决这个问题？

Answer 1

显然textcat期望使用GoldParse创建的黄金值，而不是明文值。工作版本如下所示：

import spacy
from spacy.pipeline import TextCategorizer
from spacy.gold import GoldParse
nlp = spacy.load('en')

doc1 = nlp(u'This is my first document in the dataset.')
doc2 = nlp(u'This is my second document in the dataset.')

gold1 = GoldParse(doc=doc1, cats={'Category1': 1, 'Category2': 0})
gold2 = GoldParse(doc=doc2, cats={'Category1': 0, 'Category2': 1})

textcat = TextCategorizer(nlp.vocab)
textcat.add_label('Category1')
textcat.add_label('Category2')
losses = {}
optimizer = textcat.begin_training()
textcat.update([doc1, doc2], [gold1, gold2], losses=losses, sgd=optimizer)

感谢@abarnert在评论中帮我调试这个。

spaCy分类器：＆＃39; unicode＆＃39;对象没有属性＆＃39; to_array＆＃39;

1 个答案: