安装scrapy时遇到麻烦(“分段故障(核心已转储)”)

时间:2018-09-14 13:33:12

标签: python scrapy

使用和不使用sudo命令安装scrapy时遇到问题。 scrapy -V继续返回“找不到命令”。有人认识到这个错误吗?我正在使用Lubuntu 18.04,Python 2.7.15rc1,pip 9.0.1

当我尝试安装Scrapy时,输出如下:

owner@G700:~/PhpstormProjects/yelp_scraper_1-master/yelp_scraper$ pip install Scrapy
Collecting Scrapy
  Using cached https://files.pythonhosted.org/packages/5d/12/a6197eaf97385e96fd8ec56627749a6229a9b3178ad73866a0b1fb377379/Scrapy-1.5.1-py2.py3-none-any.whl
Collecting queuelib (from Scrapy)
  Using cached https://files.pythonhosted.org/packages/4c/85/ae64e9145f39dd6d14f8af3fa809a270ef3729f3b90b3c0cf5aa242ab0d4/queuelib-1.5.0-py2.py3-none-any.whl
Collecting w3lib>=1.17.0 (from Scrapy)
  Using cached https://files.pythonhosted.org/packages/37/94/40c93ad0cadac0f8cb729e1668823c71532fd4a7361b141aec535acb68e3/w3lib-1.19.0-py2.py3-none-any.whl
Collecting pyOpenSSL (from Scrapy)
  Using cached https://files.pythonhosted.org/packages/96/af/9d29e6bd40823061aea2e0574ccb2fcf72bfd6130ce53d32773ec375458c/pyOpenSSL-18.0.0-py2.py3-none-any.whl
Collecting lxml (from Scrapy)
  Using cached https://files.pythonhosted.org/packages/e5/14/f4343239f955442da9da1919a99f7311bc5627522741bada61b2349c8def/lxml-4.2.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting parsel>=1.1 (from Scrapy)
  Using cached https://files.pythonhosted.org/packages/fd/1a/9642a5ea68763d5e7c419df0873073e54bb23d0a8897d3c78e146dd6f355/parsel-1.5.0-py2.py3-none-any.whl
Collecting service-identity (from Scrapy)
  Using cached https://files.pythonhosted.org/packages/29/fa/995e364220979e577e7ca232440961db0bf996b6edaf586a7d1bd14d81f1/service_identity-17.0.0-py2.py3-none-any.whl
Collecting six>=1.5.2 (from Scrapy)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting Twisted>=13.1.0 (from Scrapy)
Collecting PyDispatcher>=2.0.5 (from Scrapy)
Collecting cssselect>=0.9 (from Scrapy)
  Using cached https://files.pythonhosted.org/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl
Collecting cryptography>=2.2.1 (from pyOpenSSL->Scrapy)
  Using cached https://files.pythonhosted.org/packages/87/e6/915a482dbfef98bbdce6be1e31825f591fc67038d4ee09864c1d2c3db371/cryptography-2.3.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting functools32; python_version < "3.0" (from parsel>=1.1->Scrapy)
Collecting pyasn1 (from service-identity->Scrapy)
  Using cached https://files.pythonhosted.org/packages/d1/a1/7790cc85db38daa874f6a2e6308131b9953feb1367f2ae2d1123bb93a9f5/pyasn1-0.4.4-py2.py3-none-any.whl
Collecting attrs (from service-identity->Scrapy)
  Using cached https://files.pythonhosted.org/packages/3a/e1/5f9023cc983f1a628a8c2fd051ad19e76ff7b142a0faf329336f9a62a514/attrs-18.2.0-py2.py3-none-any.whl
Collecting pyasn1-modules (from service-identity->Scrapy)
  Using cached https://files.pythonhosted.org/packages/19/02/fa63f7ba30a0d7b925ca29d034510fc1ffde53264b71b4155022ddf3ab5d/pyasn1_modules-0.2.2-py2.py3-none-any.whl
Collecting PyHamcrest>=1.9.0 (from Twisted>=13.1.0->Scrapy)
  Using cached https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl
Collecting zope.interface>=4.4.2 (from Twisted>=13.1.0->Scrapy)
Collecting hyperlink>=17.1.1 (from Twisted>=13.1.0->Scrapy)
  Using cached https://files.pythonhosted.org/packages/a7/b6/84d0c863ff81e8e7de87cff3bd8fd8f1054c227ce09af1b679a8b17a9274/hyperlink-18.0.0-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from Twisted>=13.1.0->Scrapy)
  Using cached https://files.pythonhosted.org/packages/a3/86/14c16bb98a5a3542ed8fed5d74fb064a902de3bdd98d6584b34553353c45/Automat-0.7.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from Twisted>=13.1.0->Scrapy)
  Using cached https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl
Collecting constantly>=15.1 (from Twisted>=13.1.0->Scrapy)
  Using cached https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3" (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
  Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting idna>=2.1 (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
  Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.7 (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
  Using cached https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting ipaddress; python_version < "3" (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
  Using cached https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl
Collecting setuptools (from PyHamcrest>=1.9.0->Twisted>=13.1.0->Scrapy)
  Using cached https://files.pythonhosted.org/packages/66/e8/570bb5ca88a8bcd2a1db9c6246bb66615750663ffaaeada95b04ffe74e12/setuptools-40.2.0-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.7->cryptography>=2.2.1->pyOpenSSL->Scrapy)
Installing collected packages: queuelib, six, w3lib, enum34, idna, asn1crypto, pycparser, cffi, ipaddress, cryptography, pyOpenSSL, lxml, functools32, cssselect, parsel, pyasn1, attrs, pyasn1-modules, service-identity, setuptools, PyHamcrest, zope.interface, hyperlink, Automat, incremental, constantly, Twisted, PyDispatcher, Scrapy
Successfully installed Automat-0.7.0 PyDispatcher-2.0.5 PyHamcrest-1.9.0 Scrapy-1.5.1 Twisted-18.7.0 asn1crypto-0.24.0 attrs-18.2.0 cffi-1.11.5 constantly-15.1.0 cryptography-2.3.1 cssselect-1.0.3 enum34-1.1.6 functools32-3.2.3.post2 hyperlink-18.0.0 idna-2.7 incremental-17.5.0 ipaddress-1.0.22 lxml-4.2.5 parsel-1.5.0 pyOpenSSL-18.0.0 pyasn1-0.4.4 pyasn1-modules-0.2.2 pycparser-2.18 queuelib-1.5.0 service-identity-17.0.0 setuptools-40.2.0 six-1.11.0 w3lib-1.19.0 zope.interface-4.5.0
Segmentation fault (core dumped)

使用sudo:

owner@G700:~/PhpstormProjects/yelp_scraper_1-master/yelp_scraper$ sudo pip install Scrapy
The directory '/home/owner/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/owner/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: Scrapy in /home/owner/.local/lib/python2.7/site-packages
Requirement already satisfied: queuelib in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: w3lib>=1.17.0 in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: cssselect>=0.9 in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: lxml in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: parsel>=1.1 in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: service-identity in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: six>=1.5.2 in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: Twisted>=13.1.0 in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: PyDispatcher>=2.0.5 in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: pyOpenSSL in /home/owner/.local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied: functools32; python_version < "3.0" in /home/owner/.local/lib/python2.7/site-packages (from parsel>=1.1->Scrapy)
Requirement already satisfied: pyasn1 in /home/owner/.local/lib/python2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: attrs in /home/owner/.local/lib/python2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: pyasn1-modules in /home/owner/.local/lib/python2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: PyHamcrest>=1.9.0 in /home/owner/.local/lib/python2.7/site-packages (from Twisted>=13.1.0->Scrapy)
Requirement already satisfied: zope.interface>=4.4.2 in /home/owner/.local/lib/python2.7/site-packages (from Twisted>=13.1.0->Scrapy)
Requirement already satisfied: hyperlink>=17.1.1 in /home/owner/.local/lib/python2.7/site-packages (from Twisted>=13.1.0->Scrapy)
Requirement already satisfied: Automat>=0.3.0 in /home/owner/.local/lib/python2.7/site-packages (from Twisted>=13.1.0->Scrapy)
Requirement already satisfied: incremental>=16.10.1 in /home/owner/.local/lib/python2.7/site-packages (from Twisted>=13.1.0->Scrapy)
Requirement already satisfied: constantly>=15.1 in /home/owner/.local/lib/python2.7/site-packages (from Twisted>=13.1.0->Scrapy)
Requirement already satisfied: cryptography>=2.2.1 in /home/owner/.local/lib/python2.7/site-packages (from pyOpenSSL->Scrapy)
Requirement already satisfied: setuptools in /home/owner/.local/lib/python2.7/site-packages (from PyHamcrest>=1.9.0->Twisted>=13.1.0->Scrapy)
Requirement already satisfied: idna>=2.5 in /home/owner/.local/lib/python2.7/site-packages (from hyperlink>=17.1.1->Twisted>=13.1.0->Scrapy)
Requirement already satisfied: enum34; python_version < "3" in /home/owner/.local/lib/python2.7/site-packages (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
Requirement already satisfied: asn1crypto>=0.21.0 in /home/owner/.local/lib/python2.7/site-packages (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
Requirement already satisfied: cffi!=1.11.3,>=1.7 in /home/owner/.local/lib/python2.7/site-packages (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
Requirement already satisfied: ipaddress; python_version < "3" in /home/owner/.local/lib/python2.7/site-packages (from cryptography>=2.2.1->pyOpenSSL->Scrapy)
Requirement already satisfied: pycparser in /home/owner/.local/lib/python2.7/site-packages (from cffi!=1.11.3,>=1.7->cryptography>=2.2.1->pyOpenSSL->Scrapy)

尽管我是Python的新手,但我在其他任何库中都没有遇到这个问题。我还安装了3.6,想知道是否可能存在冲突?

4 个答案:

答案 0 :(得分:0)

您可以尝试更新您的点子版本吗?

最新的是pip 18.0

答案 1 :(得分:0)

使用此命令更新您的点值

sudo apt install python3-pip

答案 2 :(得分:0)

尽管我的Twisted包装是刮板的,但今天也遇到了同样的问题。这就是我解决的方法。首先要确保在您的设备上安装了pip3

sudo apt install pip3

接下来我要做的是使用pipenv创建一个虚拟环境。如果您还没有这个,那就

pip3 install pipenv

下一个cd进入您要使用的目录并创建虚拟环境

python3 -m pipenv --python 3.6 

在运行时,它将默认使用默认的可用 python 3.6版创建环境。激活您的虚拟环境

pipenv shell

最后

pip install scrapy

您这次不必添加 3 来点子,因为它已经在虚拟环境中,只有您用来创建它的python版本才可用。有了这个,你应该很好了。

答案 3 :(得分:0)

只需尝试命令:

$pip3 install scrapy 

它将起作用:)