Question

我正在尝试在Microsoft Azure Web App上运行Scrapy或Portia。我通过创建虚拟环境安装了Scrapy：

D:\Python27\Scripts\virtualenv.exe D:\home\Python

然后安装了Scrapy：

D:\home\Python\Scripts\pip install Scrapy

安装似乎有效。但是执行一个蜘蛛会返回以下输出：

D:\home\Python\Scripts\tutorial>d:\home\python\scripts\scrapy.exe crawl example 2015-09-13 23:09:31 [scrapy] INFO: Scrapy 1.0.3 started (bot: tutorial)

2015-09-13 23:09:31 [scrapy] INFO: Optional features available: ssl, http11

2015-09-13 23:09:31 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutorial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'BOT_NAME': 'tutorial'}

2015-09-13 23:09:34 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsole, LogStats, CoreStats, SpiderState

Unhandled error in Deferred:

2015-09-13 23:09:35 [twisted] CRITICAL: Unhandled error in Deferred:

Traceback (most recent call last):

  File "D:\home\Python\lib\site-packages\scrapy\cmdline.py", line 150, in _run_command

    cmd.run(args, opts)

  File "D:\home\Python\lib\site-packages\scrapy\commands\crawl.py", line 57, in run

    self.crawler_process.crawl(spname, **opts.spargs)

  File "D:\home\Python\lib\site-packages\scrapy\crawler.py", line 153, in crawl

    d = crawler.crawl(*args, **kwargs)

  File "D:\home\Python\lib\site-packages\twisted\internet\defer.py", line 1274, in unwindGenerator

    return _inlineCallbacks(None, gen, Deferred())

--- <exception caught here> ---

  File "D:\home\Python\lib\site-packages\twisted\internet\defer.py", line 1128, in _inlineCallbacks

    result = g.send(result)

  File "D:\home\Python\lib\site-packages\scrapy\crawler.py", line 71, in crawl

    self.engine = self._create_engine()

  File "D:\home\Python\lib\site-packages\scrapy\crawler.py", line 83, in _create_engine

    return ExecutionEngine(self, lambda _: self.stop())

  File "D:\home\Python\lib\site-packages\scrapy\core\engine.py", line 66, in __init__

    self.downloader = downloader_cls(crawler)

  File "D:\home\Python\lib\site-packages\scrapy\core\downloader\__init__.py", line 65, in __init__

    self.handlers = DownloadHandlers(crawler)

  File "D:\home\Python\lib\site-packages\scrapy\core\downloader\handlers\__init__.py", line 23, in __init__

    cls = load_object(clspath)

  File "D:\home\Python\lib\site-packages\scrapy\utils\misc.py", line 44, in load_object

    mod = import_module(module)

  File "D:\Python27\Lib\importlib\__init__.py", line 37, in import_module

    __import__(name)

  File "D:\home\Python\lib\site-packages\scrapy\core\downloader\handlers\s3.py", line 6, in <module>

    from .http import HTTPDownloadHandler

  File "D:\home\Python\lib\site-packages\scrapy\core\downloader\handlers\http.py", line 5, in <module>

    from .http11 import HTTP11DownloadHandler as HTTPDownloadHandler

  File "D:\home\Python\lib\site-packages\scrapy\core\downloader\handlers\http11.py", line 15, in <module>

    from scrapy.xlib.tx import Agent, ProxyAgent, ResponseDone, \

  File "D:\home\Python\lib\site-packages\scrapy\xlib\tx\__init__.py", line 3, in <module>

    from twisted.web import client

  File "D:\home\Python\lib\site-packages\twisted\web\client.py", line 42, in <module>

    from twisted.internet.endpoints import TCP4ClientEndpoint, SSL4ClientEndpoint

  File "D:\home\Python\lib\site-packages\twisted\internet\endpoints.py", line 34, in <module>

    from twisted.internet.stdio import StandardIO, PipeAddress

  File "D:\home\Python\lib\site-packages\twisted\internet\stdio.py", line 30, in <module>

    from twisted.internet import _win32stdio

  File "D:\home\Python\lib\site-packages\twisted\internet\_win32stdio.py", line 7, in <module>

    import win32api

exceptions.ImportError: No module named win32api



2015-09-13 23:09:35 [twisted] CRITICAL:

文档http://doc.scrapy.org/en/latest/intro/install.html说我必须安装 pywin32 。我不知道如何通过命令行下载/安装它，因为我在网络应用程序环境中。

甚至可以在Azure Web App上运行Scrapy或Portia，还是必须在Azure上使用完全成熟的虚拟机？

谢谢！

Answer 1

您无法在Azure Web App上“运行”通用Windows应用程序。作为Web应用程序在Azure上运行的东西必须专门构建才能这样做。因此，您必须在Azure上使用完整的虚拟机。

Azure Webapps似乎可以运行一些Python应用程序，如果它们构建在某些框架上： Jade

如何在Azure Web App上运行Scrapy / Portia

1 个答案: