我有一个简单的光束管道,可以在其中读取pub sub中的数据并将其写入文件。我在直接运行程序上运行它,代码如下:
pubsub_data = (
p
| 'Read from pub sub' >> beam.io.ReadFromPubSub(topic = '{my-topic}')
| 'Write to a file' >> beam.io.WriteToText(output_file)
)
但是,该作业失败,并由于错误导入而导致堆栈跟踪,如下所示。
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/executor.py", line 343, in call
finish_state)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/executor.py", line 368, in attempt_call
side_input_values)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/transform_evaluator.py", line 116, in get_evaluator
input_committed_bundle, side_inputs)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/transform_evaluator.py", line 433, in __init__
self.source.subscription_name)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/transform_evaluator.py", line 438, in get_subscription
wrapper = _PubSubSubscriptionWrapper(project, topic, short_sub_name)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/transform_evaluator.py", line 396, in __init__
from google.cloud import pubsub
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub.py", line 19, in <module>
from google.cloud.pubsub_v1 import PublisherClient
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
from google.cloud.pubsub_v1 import types
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/types.py", line 20, in <module>
from google.iam.v1 import iam_policy_pb2
ImportError: No module named iam.v1
点冻结显示所需的库grpc-google-iam-v1==0.11.4
已经存在
absl-py==0.7.1
aenum==2.1.2
alabaster==0.7.12
albumentations==0.1.12
altair==3.1.0
apache-beam==2.13.0
astor==0.8.0
astropy==2.0.14
atari-py==0.1.7
atomicwrites==1.3.0
attrs==19.1.0
audioread==2.1.8
autograd==1.2
avro==1.9.0
Babel==2.7.0
backports-abc==0.5
backports.functools-lru-cache==1.5
backports.shutil-get-terminal-size==1.0.0
backports.tempfile==1.0
backports.weakref==1.0.post1
bcolz==1.2.1
beautifulsoup4==4.6.3
bleach==3.1.0
blis==0.2.4
bokeh==1.0.4
boto==2.49.0
boto3==1.9.185
botocore==1.12.185
branca==0.3.1
bs4==0.0.1
bz2file==0.98
cachetools==3.1.1
certifi==2019.6.16
cffi==1.12.3
chainer==5.4.0
chardet==3.0.4
Click==7.0
click-plugins==1.1.1
cligj==0.5.0
cloudpickle==0.6.1
cmake==3.12.0
colorlover==0.3.0
community==1.0.0b1
configparser==3.7.4
contextlib2==0.5.5
convertdate==2.1.3
crcmod==1.7
cufflinks==0.14.6
cvxopt==1.2.3
cvxpy==1.0.15
cycler==0.10.0
cymem==2.0.2
Cython==0.29.12
daft==0.0.4
dask==1.1.5
decorator==4.4.0
defusedxml==0.6.0
descartes==1.1.0
dill==0.2.9
distributed==1.25.3
Django==1.11.22
dlib==19.16.0
dm-sonnet==1.34
docopt==0.6.2
docutils==0.14
dopamine-rl==1.0.5
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.3
en-core-web-sm==2.1.0
entrypoints==0.3
enum34==1.1.6
ephem==3.7.6.0
et-xmlfile==1.0.1
fa2==0.3.5
fancyimpute==0.4.3
fastai==0.7.0
fastavro==0.21.24
fastcache==1.1.0
fastdtw==0.3.2
fasteners==0.15
fastrlock==0.4
fbprophet==0.5
feather-format==0.4.0
featuretools==0.4.1
filelock==3.0.12
Fiona==1.8.6
fix-yahoo-finance==0.0.22
Flask==1.1.1
folium==0.8.3
funcsigs==1.0.2
functools32==3.2.3.post2
future==0.16.0
futures==3.2.0
gast==0.2.2
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.49
geopandas==0.5.0
geopy==1.17.0
gevent==1.4.0
gin-config==0.1.4
glob2==0.7
google==2.0.2
google-api-core==1.13.0
google-api-python-client==1.7.9
google-apitools==0.5.28
google-auth==1.4.2
google-auth-httplib2==0.0.3
google-auth-oauthlib==0.4.0
google-cloud-bigquery==1.6.1
google-cloud-bigtable==0.32.2
google-cloud-core==0.29.1
google-cloud-datastore==1.7.4
google-cloud-language==1.2.0
google-cloud-pubsub==0.39.1
google-cloud-storage==1.16.1
google-cloud-translate==1.5.0
google-colab==1.0.0
google-pasta==0.1.7
google-resumable-media==0.3.2
googleapis-common-protos==1.6.0
googledatastore==7.0.2
googledrivedownloader==0.4
graph-nets==1.0.4
graphviz==0.10.1
greenlet==0.4.15
grpc-google-iam-v1==0.11.4
grpcio==1.15.0
gspread==3.0.1
gspread-dataframe==3.0.2
gunicorn==19.9.0
gym==0.10.11
h5py==2.8.0
hdfs==2.5.8
HeapDict==1.0.0
holidays==0.9.10
html5lib==1.0.1
httpimport==0.5.16
httplib2==0.11.3
humanize==0.5.1
hyperopt==0.1.2
ideep4py==2.0.0.post3
idna==2.8
image==1.5.27
imageio==2.4.1
imagesize==1.1.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.2.9
imutils==0.5.2
inflect==2.1.0
intel-openmp==2019.0
intervaltree==2.1.0
ipaddress==1.0.22
ipykernel==4.6.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.5.0
isoweek==1.3.3
itsdangerous==1.1.0
jdcal==1.4.1
jedi==0.14.0
jieba==0.39
Jinja2==2.10.1
jmespath==0.9.4
joblib==0.12.5
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.3.1
jupyter-console==5.2.0
jupyter-core==4.5.0
kaggle==1.5.4
kapre==0.1.3.1
Keras==2.2.4
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
keras-vis==0.4.1
kiwisolver==1.1.0
knnimpute==0.1.0
librosa==0.6.3
lightgbm==2.2.3
llvmlite==0.29.0
lmdb==0.95
lucid==0.3.8
lunardate==0.2.0
lxml==4.2.6
magenta==0.3.19
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==2.2.4
matplotlib-venn==0.11.5
mesh-tensorflow==0.0.5
mido==1.2.6
mir-eval==0.5
missingno==0.4.2
mistune==0.8.4
mizani==0.4.6
mkl==2019.0
mlxtend==0.14.0
mock==2.0.0
monotonic==1.5
more-itertools==5.0.0
moviepy==0.2.3.5
mpmath==1.1.0
msgpack==0.5.6
multiprocess==0.70.8
multitasking==0.0.9
munch==2.3.2
murmurhash==1.0.2
music21==4.1.0
natsort==5.5.0
nbconvert==5.5.0
nbformat==4.4.0
networkx==2.2
nibabel==2.3.3
nltk==3.2.5
nose==1.3.7
notebook==5.2.2
np-utils==0.5.10.0
numba==0.40.1
numexpr==2.6.9
numpy==1.16.4
oauth2client==3.0.0
oauthlib==3.0.2
okgrade==0.4.3
olefile==0.46
opencv-contrib-python==3.4.3.18
opencv-python==3.4.5.20
openpyxl==2.5.9
osqp==0.5.0
packaging==19.0
palettable==3.1.1
pandas==0.24.2
pandas-datareader==0.7.0
pandas-gbq==0.4.1
pandas-profiling==1.4.1
pandas-summary==0.0.6
pandocfilters==1.4.2
parso==0.5.0
pathlib==1.0.1
pathlib2==2.3.4
patsy==0.5.1
pbr==5.4.0
pexpect==4.7.0
pickleshare==0.7.5
Pillow==4.3.0
pip-tools==3.6.1
plac==0.9.6
plotly==3.6.1
plotnine==0.4.0
pluggy==0.7.1
portpicker==1.2.0
prefetch-generator==1.0.1
preshed==2.0.1
pretty-midi==0.2.8
prettytable==0.7.2
progressbar2==3.38.0
prometheus-client==0.7.1
promise==2.2.1
prompt-toolkit==1.0.16
proto-google-cloud-datastore-v1==0.90.4
protobuf==3.7.1
psutil==5.4.8
psycopg2==2.7.6.1
ptyprocess==0.6.0
py==1.8.0
pyarrow==0.13.0
pyasn1==0.4.5
pyasn1-modules==0.2.5
pycocotools==2.0.0
pycparser==2.19
pydot==1.2.4
pydot-ng==2.0.0
pydotplus==2.0.2
pyemd==0.5.1
pyglet==1.4.1
Pygments==2.1.3
pymc3==3.6
pymongo==3.8.0
pymystem3==0.2.0
PyOpenGL==3.1.0
pyparsing==2.4.0
pyproj==2.2.1
pyrsistent==0.15.3
pysndfile==1.3.3
PySocks==1.7.0
pystan==2.19.0.0
pytest==3.6.4
python-chess==0.23.11
python-dateutil==2.5.3
python-louvain==0.13
python-rtmidi==1.3.0
python-slugify==3.0.2
python-utils==2.3.0
pytz==2018.9
PyVCF==0.6.8
PyWavelets==1.0.3
PyYAML==3.13
pyzmq==17.0.0
qtconsole==4.5.1
requests==2.21.0
requests-oauthlib==1.2.0
resampy==0.2.1
retrying==1.3.3
rpy2==2.8.6
rsa==4.0
s3fs==0.2.2
s3transfer==0.2.1
scandir==1.10.0
scikit-image==0.14.3
scikit-learn==0.20.3
scipy==1.2.2
scs==2.1.1.post2
seaborn==0.9.0
semantic-version==2.6.0
Send2Trash==1.5.0
setuptools-git==1.2
Shapely==1.6.4.post2
simplegeneric==0.8.1
singledispatch==3.4.0.3
siphash==0.0.1
six==1.12.0
sklearn==0.0
sklearn-pandas==1.8.0
smart-open==1.8.4
snowballstemmer==1.9.0
sortedcontainers==2.1.0
spacy==2.1.4
Sphinx==1.8.5
sphinxcontrib-websupport==1.1.2
SQLAlchemy==1.3.5
sqlparse==0.3.0
srsly==0.0.7
statsmodels==0.10.0
subprocess32==3.5.4
sympy==1.1.1
tables==3.4.4
tabulate==0.8.3
tblib==1.4.0
tensor2tensor==1.11.0
tensorboard==1.14.0
tensorboardcolab==0.0.22
tensorflow==1.14.0
tensorflow-estimator==1.14.0
tensorflow-hub==0.5.0
tensorflow-metadata==0.13.0
tensorflow-probability==0.7.0
termcolor==1.1.0
terminado==0.8.2
testpath==0.4.2
text-unidecode==1.2
textblob==0.15.3
tfds-nightly==1.0.2.dev201907090105
tflearn==0.3.2
Theano==1.0.4
thinc==7.0.4
toolz==0.9.0
torch==1.1.0
torchsummary==1.5.1
torchtext==0.3.1
torchvision==0.3.0
tornado==4.5.3
tqdm==4.28.1
traitlets==4.3.2
tweepy==3.6.0
typing==3.6.6
tzlocal==1.5.1
umap-learn==0.3.9
uritemplate==3.0.0
urllib3==1.24.3
vega-datasets==0.7.0
wasabi==0.2.2
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.15.4
widgetsnbextension==3.5.0
wordcloud==1.5.0
wrapt==1.11.2
xarray==0.11.3
xgboost==0.82
xlrd==1.1.0
xlwt==1.3.0
yellowbrick==0.9.1
zict==1.0.0
如何解决此问题?