Question

我正在尝试使用BERT在Python中进行命名实体识别，并使用pip install transformers从huggestface安装了变形金刚v 3.0.2 。然后，当我尝试运行此代码时：

import torch
from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler
from transformers import BertTokenizer, BertConfig

from keras.preprocessing.sequence import pad_sequences
from sklearn.model_selection import train_test_split

torch.__version__

我收到此错误：

ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so, 2): Symbol not found: ____chkstk_darwin
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libSystem.B.dylib
 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so

此行中发生错误：from transformers import BertTokenizer, BertConfig，但我不确定如何解决此问题。

Answer 1

尝试pip install tokenizers==0.7.0

Python ImportError：从转换器导入BertTokenizer，BertConfig

1 个答案: