Question

我是第一个将自定义magics加载到ipython的新手。

的加载说明

sparkmagics

扩展程序包括以下已成功完成的步骤：

pip install sparkmagic
pip show sparkmagic
jupyter-kernelspec install sparkmagic/kernels/sparkkernel
jupyter-kernelspec install sparkmagic/kernels/pysparkkernel
jupyter-kernelspec install sparkmagic/kernels/pyspark3kernel
jupyter-kernelspec install sparkmagic/kernels/sparkrkernel

命令pip show sparkmagic似乎是健康的：

$pip show sparkmagic
Name: sparkmagic
Version: 0.10.1
Summary: SparkMagic: Spark execution via Livy
Home-page: https://github.com/jupyter-incubator/sparkmagic/sparkmagic
Author: Jupyter Development Team
Author-email: jupyter@googlegroups.org
License: BSD 3-clause
Location: /usr/local/lib/python2.7/site-packages
Requires: mock, requests, tornado, nose, notebook, ipywidgets, ipykernel, pandas, hdijupyterutils, autovizwidget, ipython, numpy

不工作的东西 - 但列为可选项;

$ jupyter serverextension enable --py sparkmagic

Traceback (most recent call last):
  File "/usr/local/bin/jupyter-serverextension", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/jupyter_core/application.py", line 267, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/usr/local/lib/python3.6/site-packages/notebook/serverextensions.py", line 300, in start
    super(ServerExtensionApp, self).start()
  File "/usr/local/lib/python3.6/site-packages/jupyter_core/application.py", line 256, in start
    self.subapp.start()
  File "/usr/local/lib/python3.6/site-packages/notebook/serverextensions.py", line 217, in start
    self.toggle_server_extension_python(arg)
  File "/usr/local/lib/python3.6/site-packages/notebook/serverextensions.py", line 206, in toggle_server_extension_python
    m, server_exts = _get_server_extension_metadata(package)
  File "/usr/local/lib/python3.6/site-packages/notebook/serverextensions.py", line 334, in _get_server_extension_metadata
    m = import_item(module)
  File "/usr/local/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
    return __import__(parts[0])
ModuleNotFoundError: No module named 'sparkmagic'

错误ModuleNotFoundError: No module named 'sparkmagic'似乎不祥。鉴于show sparkmagic健康，那么为何投诉呢？ 正确设置 sparkmagic 模块还缺少什么？

无论如何，我也在ipython里面尝试过，并且可以预见它失败了：

%load_ext sparkmagic.magics

这是（非常长）堆栈跟踪的第一部分。如果需要整个跟踪，请告诉我。

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-2b579b518c56> in <module>()
----> 1 get_ipython().magic(u'load_ext sparkmagic.magics')

/usr/local/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in magic(self, arg_s)
   2161         magic_name, _, magic_arg_s = arg_s.partition(' ')
   2162         magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2163         return self.run_line_magic(magic_name, magic_arg_s)
   2164
   2165     #-------------------------------------------------------------------------

/usr/local/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line)
   2082                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2083             with self.builtin_trap:
-> 2084                 result = fn(*args,**kwargs)
   2085             return result
   2086

<decorator-gen-64> in load_ext(self, module_str)

/usr/local/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
    191     # but it's overkill for just that one bit of state.
    192     def magic_deco(arg):
--> 193         call = lambda f, *a, **k: f(*a, **k)
    194
    195         if callable(arg):

/usr/local/lib/python2.7/site-packages/IPython/core/magics/extension.pyc in load_ext(self, module_str)
     64         if not module_str:
     65             raise UsageError('Missing module name.')
---> 66         res = self.shell.extension_manager.load_extension(module_str)
     67
     68         if res == 'already loaded':

总之，安装/使用任何魔法通常需要一个步骤。指针赞赏。

Answer 1

从我在错误中看到并安装消息，你为python2.7安装了spark magic并试图在python 3.6中运行它。令人困惑的是，当我开始时，我也多次遇到同样的问题。我建议您通过在笔记本上运行!python --version来检查您正在使用的python版本。如果它是正确版本的python，那么使用!pip install sparkmagic安装sparkmagic并检查安装消息是否安装到正确版本的python。

您还可以设置环境，让生活更轻松。更多https://www.youtube.com/watch?v=LjQlmee58hg

无法加载ipython magic扩展名sparkmagic：找不到模块

1 个答案: