Question

我正在尝试使用自制的python版本安装Scrapy。尽管有最新的，安装的和我的路径，我不能使用它。

这是我的操作顺序：

检查brew：

$ brew doctor
Your system is ready to brew.

检查python＆amp; PIP：

$ which python
/usr/local/bin/python
$ python --version
Python 2.7.6
$ which pip
/usr/local/bin/pip
$ pip -- version
pip 1.5.4 from /usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)

（这是正常的吗？看起来有两个副本，可能有冲突）

我已经安装了scrapy但是当我再次尝试时，我得到：

$ pip install scrapy
Requirement already satisfied (use --upgrade to upgrade): scrapy in /usr/local/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): Twisted>=10.0.0 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.2 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): queuelib in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0 in /usr/local/lib/python2.7/site-packages (from Twisted>=10.0.0->scrapy)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg (from zope.interface>=3.6.0->Twisted>=10.0.0->scrapy)
Cleaning up...

这对我来说很奇怪，因为我希望我的模块安装在/usr/local/share/python而不是/usr/local/lib/python2.7/site-packages 果然，/usr/local/lib/python2.7/site-packages中没有scrapy，但/usr/local/share/python中有scrapy。

我已将/usr/local/share/python/scrapy添加到我的路径中，如下所述：Scrapy installation on OSX Lion

但which scrapy返回空白。

最有趣的部分是当我尝试使用pip install --upgrade scrapy'更新'scrapy时：

~ ∮ pip install --upgrade scrapy
Requirement already up-to-date: scrapy in /usr/local/lib/python2.7/site-packages
Requirement already up-to-date: Twisted>=10.0.0 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already up-to-date: w3lib>=1.2 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already up-to-date: queuelib in /usr/local/lib/python2.7/site-packages (from scrapy)
Downloading/unpacking lxml from https://pypi.python.org/packages/source/l/lxml/lxml-3.3.2.tar.gz#md5=a3ea7bf74b718ecb46d9fd5198eec92d (from scrapy)
  Downloading lxml-3.3.2.tar.gz (3.5MB): 3.5MB downloaded
  Running setup.py (path:/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/lxml/setup.py) egg_info for package lxml
    /usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.3.2.
    Building without Cython.
    Using build configuration of libxslt 1.1.28

    warning: no previously-included files found matching '*.py'
Downloading/unpacking pyOpenSSL from https://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.14.tar.gz#md5=8579ff3a1d858858acfba5f046a4ddf7 (from scrapy)
  Downloading pyOpenSSL-0.14.tar.gz (128kB): 128kB downloaded
  Running setup.py (path:/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/pyOpenSSL/setup.py) egg_info for package pyOpenSSL

    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    no previously-included directories found matching 'doc/_build'
Requirement already up-to-date: cssselect>=0.9 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already up-to-date: six>=1.5.2 in /usr/local/lib/python2.7/site-packages (from scrapy)
Requirement already up-to-date: zope.interface>=3.6.0 in /usr/local/lib/python2.7/site-packages (from Twisted>=10.0.0->scrapy)
Downloading/unpacking cryptography>=0.2.1 (from pyOpenSSL->scrapy)
  Downloading cryptography-0.2.1.tar.gz (13.8MB): 13.8MB downloaded
  Running setup.py (path:/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py) egg_info for package cryptography
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found

    Installed /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/cffi-0.8.1-py2.7-macosx-10.9-x86_64.egg
    Searching for pycparser
    Reading https://pypi.python.org/simple/pycparser/
    Best match: pycparser 2.10
    Downloading https://pypi.python.org/packages/source/p/pycparser/pycparser-2.10.tar.gz#md5=d87aed98c8a9f386aa56d365fe4d515f
    Processing pycparser-2.10.tar.gz
    Writing /var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/easy_install-qqW3QR/pycparser-2.10/setup.cfg
    Running pycparser-2.10/setup.py -q bdist_egg --dist-dir /var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/easy_install-qqW3QR/pycparser-2.10/egg-dist-tmp-pniw4C
    zip_safe flag not set; analyzing archive contents...

    Installed /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/pycparser-2.10-py2.7.egg

    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py", line 113, in <module>
        "build": cffi_build,
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup
        dist.run_commands()
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "<string>", line 15, in replacement_run
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 186, in find_sources
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 246, in run
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 282, in add_defaults
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/sdist.py", line 167, in add_defaults
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
        cmd_obj.ensure_finalized()
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
        self.finalize_options()
      File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/build_py.py", line 26, in finalize_options
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options
        ('force', 'force'))
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options
        src_cmd_obj.ensure_finalized()
      File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
        self.finalize_options()
      File "/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py", line 52, in finalize_options
        from cryptography.hazmat.primitives import constant_time, padding
      File "cryptography/hazmat/primitives/constant_time.py", line 18, in <module>
        import six
    ImportError: No module named six
    Complete output from command python setup.py egg_info:
    Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found

Package libffi was not found in the pkg-config search path.

Perhaps you should add the directory containing `libffi.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libffi' found



Installed /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/cffi-0.8.1-py2.7-macosx-10.9-x86_64.egg

Searching for pycparser

Reading https://pypi.python.org/simple/pycparser/

Best match: pycparser 2.10

Downloading https://pypi.python.org/packages/source/p/pycparser/pycparser-2.10.tar.gz#md5=d87aed98c8a9f386aa56d365fe4d515f

Processing pycparser-2.10.tar.gz

Writing /var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/easy_install-qqW3QR/pycparser-2.10/setup.cfg

Running pycparser-2.10/setup.py -q bdist_egg --dist-dir /var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/easy_install-qqW3QR/pycparser-2.10/egg-dist-tmp-pniw4C

zip_safe flag not set; analyzing archive contents...



Installed /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/pycparser-2.10-py2.7.egg

running egg_info

creating pip-egg-info/cryptography.egg-info

writing requirements to pip-egg-info/cryptography.egg-info/requires.txt

writing pip-egg-info/cryptography.egg-info/PKG-INFO

writing top-level names to pip-egg-info/cryptography.egg-info/top_level.txt

writing dependency_links to pip-egg-info/cryptography.egg-info/dependency_links.txt

writing manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found



Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py", line 113, in <module>

    "build": cffi_build,

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 152, in setup

    dist.run_commands()

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands

    self.run_command(cmd)

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command

    cmd_obj.run()

  File "<string>", line 15, in replacement_run

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 186, in find_sources

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 246, in run

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/egg_info.py", line 282, in add_defaults

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/sdist.py", line 167, in add_defaults

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command

    cmd_obj.ensure_finalized()

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized

    self.finalize_options()

  File "build/bdist.macosx-10.9-x86_64/egg/setuptools/command/build_py.py", line 26, in finalize_options

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options

    ('force', 'force'))

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options

    src_cmd_obj.ensure_finalized()

  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized

    self.finalize_options()

  File "/private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_pauluslabuser/cryptography/setup.py", line 52, in finalize_options

    from cryptography.hazmat.primitives import constant_time, padding

  File "cryptography/hazmat/primitives/constant_time.py", line 18, in <module>

    import six

ImportError: No module named six

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /private/var/folders/22/6lh7b3m53zx1xslwtkdk5v400000gp/T/pip_build_myAccount/cryptography
Storing debug log for failure in /Users/myAccount/.pip/pip.log
~ ∮

我已尝试补救其中一些内容，例如pip install six，并尝试使用Google egg_info failed with error code 1进行搜索。大多数解决方案都没有安装setuptools，但是：

$ pip install setuptools
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg
Cleaning up...

它看起来像我的brew安装python和不正确的链接/多个版本？ pip导致模块安装在一个地方，而我的python在其他地方（这解释了为什么>>> import scrapy失败但不解释为什么$ scrapy失败）。

关于损坏的python模块的类似帖子建议运行xcode-select --install我已经确认是最新的。 Can't install PIL after Mac OS X 10.9

也许是它的虚拟时间？

提前感谢！

Answer 1

这对我有用：

由于您使用Homebrew作为软件包管理器，因此通过它安装的程序将设置为/usr/local/bin，而不是开箱即用的标准/usr/bin路径设置应用。您需要确保/usr/local/bin位于/etc/paths文件的顶部。该文件应该被锁定，因此您可能需要使用以下内容在vi编辑器中保存更改：
```
:w !sudo tee %
```

$ which python应返回/usr/local/bin/python，否则：

`$ brew unlink python`
`$ brew install python`
`$ brew linkapps`

$ which pip应该返回/usr/local/bin/pip，否则
```
`$ brew install pip`
```

则...

`$ sudo pip uninstall Scrapy`
`$ pip install Scrapy`
`$ pip install --upgrade Scrapy`
`$ pip install --upgrade lxml`
`$ scrapy startproject tutorial`

Answer 2

我在安装virtualenv：Can't pip install virtualenv on Mac 10.9 Mavericks with brewed python

时遇到了类似的问题

这里的解决方案也适用，虽然我仍然不明白为什么问题首先出现以及为什么这个解决方案有效。但确实如此。

卸载并重新安装scrapy：

$pip uninstall scrapy
$pip install scrapy

这会删除/usr/local/lib/python2.7/site-packages中的模块并将其安装在/usr/local/bin/

中

问题是，根据自制的python文档，pip假设安装在/site-packages中，它们应该可以工作。它不仅无法在/site-packages中与它们一起使用，而且pip默认安装在/usr/local/bin/中。

这种解决方法有效，虽然我真的很感激解释为什么它有效以及为什么这首先是一个问题。那么也许这两个问题可以合并？

Answer 3

在OS X 10.10 Yosemite中，以上都没有为我工作。

但我找到了解决方案here。

使用pkg-config：

brew install pkg-config libffi
export PKG_CONFIG_PATH=/usr/local/Cellar/libffi/3.0.13/lib/pkgconfig/
pip install cffi

您必须根据您的libffi版本更改路径。

Answer 4

您需要为安装环境提供对OpenSSL捆绑的Crypto库的访问权限。这是对我有用的解决方案。

env ARCHFLAGS =“ - arch x86_64”LDFLAGS =“ - L / usr / local / opt / openssl / lib”CFLAGS =“ - I / usr / local / opt / openssl / include”

请记住将其更改为更新的OpenSSL包的位置。

Answer 5

其他帖子对我没用，但确实如此：

sudo ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install scrapy

麻烦在10.9小牛队上安装Scrapy

5 个答案: