Question

尝试使用Scrapy shell from scrapy.shell import inspect_response inspect_response(response, self)命令

shelp()
[s] Available Scrapy objects:
[s]   scrapy     scrapy module (contains scrapy.Request, scrapy.Selector, etc)
[s]   crawler    <scrapy.crawler.Crawler object at 0x10b23ecd0>
[s]   item       {}
[s]   request    <GET https://inventory.dealersocket.com/admin/inventory/current>
[s]   response   <200 https://inventory.dealersocket.com/admin/inventory/current>
[s]   settings   <scrapy.settings.Settings object at 0x10b23ec50>
[s] Useful shortcuts:
[s]   shelp()           Shell help (print this help)
[s]   view(response)    View response in a browser

取       Traceback（最近一次调用最后一次）：         文件“”，第1行，in       NameError：名称'fetch'未定义

IBOutlets

如您所见，没有提取命令。

问题 - 如何从Scrapy shell做请求？

Answer 1

只能通过df = pd.DataFrame({'Age': np.random.rand(25) * 160}) df['Length'] = df['Age'] * 0.88 + np.random.rand(25) * 5000 fig, ax = plt.subplots() ax.scatter(df['Length'], df['Age']) slope = 0.03 x_0 = 0 y_0 = 0 x_1 = 5000 slopes = np.linspace(0.01, 0.05, 5) # create an array containing the gradients new_y = (slopes * x_1) + y_0 # find the corresponding y values at x = 5000 for i in range(len(slopes)): ax.plot([x_0, x_1], [y_0, new_y[i]], marker='^', markersize=10, label=slopes[i]) plt.legend(title="Gradients") plt.show()命令获取提取。它在爬行期间不可用，因为scrapy引擎已经忙于爬行蜘蛛，所以fetch不适合。

然而，可以通过将高优先级请求安排到临时回调来解决这个问题：

scrapy shell

当提示您class MySpider(Spider): name = 'myspider' def parse(self, response): from scrapy.shell import inspect_response inspect_response(response, self) def _fetch_parse(self, response): from scrapy.shell import inspect_response inspect_response(response, self) def fetch(url): # schedule high priority requests directly crawler.engine.schedule(Request(url, self._fetch_parse, priority=1000))中的shell时，您可以：

parse

Scrapy shell - 'fetch'未定义

1 个答案: