Question

doc.xpath('//img') #this will get some results
doc.xpath('//img[@class="productImage"]') #and this gets nothing at all
doc.xpath('//div[@id="someID"]') # and this won't work either

我不知道这里出了什么问题，我仔细检查了HTML源代码，有很多img标签包含属性（class =“productImage”）。

就像属性选择器不起作用。

以下是HTML源代码所来自的URL。

http://www.amazon.cn/s/ref=nb_sb_ss_i_0_1?__mk_zh_CN=%E4%BA%9A%E9%A9%AC%E9%80%8A%E7%BD%91%E7%AB%99&url=search-alias%3Daps&field-keywords=%E4%B8%93%E5%85%AB&x=0&y=0&sprefix=%E4%B8%93

如果你有空闲时间，请帮我一个忙。解析HTML内容，就像我看到你能解决这个问题一样

Answer 1

奇怪的是，如果您在该页面上使用open-uri，则会得到与使用curl或wget之类的内容不同的结果。

但是当您更改User-Agent时，实际上可能是您正在寻找的页面：

<强>分析：

require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'pp'  

URL = 'http://www.amazon.cn/...'

def analyze_html(file)
   doc = Nokogiri.HTML(file)
   pp   doc.xpath('//img').map { |i| i[:class] }.compact.reject(&:empty?)
   puts doc.xpath('//div').map { |i| i[:class] }.grep(/productImage/).count
   puts doc.xpath('//div[@class="productImage"]//img').count
   pp   doc.xpath('//div[@class="productImage"]//img').map { |i| i[:src] }
end

puts "Attempt 1:"
analyze_html(open(URL))

puts "Attempt 2:"
analyze_html(open(URL, "User-Agent" => "Wget/1.10.2"))

<强>输出：

Attempt 1:
["default navSprite"]
0
0
[]
Attempt 2:
["default navSprite", "srSprite spr_kindOfSortBtn"]
16
16
["http://ec4.images-amazon.com/images/I/51fOb3ujSjL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/513UQ1xiaSL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/41zKxWXb8HL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51bj6XXAouL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/516GBhDTGCL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51ADd3HSE6L._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51CbB-7kotL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51%2Bw40Mk51L._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/519Gny1LckL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51Dv6DUF-WL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51uuy8yHeoL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51T0KEjznqL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/419WTi%2BdjzL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51QTg4ZmMmL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51l--Pxw9TL._AA115_.jpg",
 "http://ec4.images-amazon.com/images/I/51gehW2qUZL._AA115_.jpg"]

<强>解决方案：

使用User-Agent: Wget/1.10.2
使用xpath('//div[@class="productImage"]//img')

Nokogiri xpath方法有些奇怪的问题

1 个答案: