xpath在开发人员控制台中找到元素,但在scrapy.response中找不到

时间:2015-07-05 19:48:55

标签: html python-2.7 xpath web-scraping scrapy

我正在尝试使用此xpath从第一张票here页面中获取价格:

'.//*[@class="price"]/text()'

这可以在开发人员的控制台中使用,但不是在我使用response.xpath在scrapy shell中运行它时。我也试着在shell中关注:

'.//*[@class="initial"]/div[@class="price"]/text()'

'//*[@id="tVB901769989"]/div[1]/div[4]'(虽然我不认为id属性可以在shell中使用)。

我使用的xpath是否有问题,或者页面的工作方式有什么不同?任何帮助,将不胜感激。谢谢!

1 个答案:

答案 0 :(得分:3)

发生这种情况是因为您正在检查不同的请求,您看到的页面没有您在该文件中所需的信息,但它会动态获取,在这种情况下来自:www.vividseats.com/javascript/tickets。 SHTML?productionId = 1771684

在那里你可以查看json格式的价格,我认为这是一个项目:

{
   "s":"Section 111",
   "r":"8-22",
   "q":"4",
   "p":"692.00",
   "i":"VB782041491",
   "d":"111",
   "n":"Zone Seating. The seller is committing to procure these tickets for you upon receipt of your order. After you place your order and your order is confirmed, we guarantee that your tickets will be within the listed zone
 or section listed or one comparable and that you will receive these tickets in time for the event or
 your money back. Orders exceeding four tickets may be split up into different rows within the requested
 zone or section.",
   "f":"0",
   "l":"Section 111",
   "g":"0",
   "e":"0",
   "h":"07/21/15",
   "t":"0",
   "v":"",
   "c":"84352",
   "z":"1",
   "rhdn":"0",
   "ind":"0",
   "sd":"0"
}

其中p包含价格。