Question

我正在使用Rails 4.2.3。我有这个代码，用于获取URL的内容

  doc = Nokogiri::HTML(open(url))

有时URL会返回HTML，但有时它会返回JSON。我不提前知道。我注意到的是，当URL返回JSON时，Nokgiri会在其前面添加所有这些HTML标记。以下是浏览器中显示的内容：

{"list":[{"u":"1459808276_000001","i":"1459184695_000001","pid":"RDE8UZZZ”,”fname":"Alexi","lname”:”Jones”,”sex":"F","city":"Eugene","country":"US","country_iso":"us","course":"8k","class":"elite","race":"8K","name":"Alexi Jones”,”_ver":"14","tag":"0000001","bib":"1"}],"info":{"first":"1","last":"1","total":"1","cacheVer":"0~0"}}

然而，当我执行Nokogiri时，会返回：

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body><p>{"list":[{"u":"1459808276_000001","i":"1459184695_000001","pid":"RDE8UZZZ”,”fname":"Alexi","lname”:”Jones”,”sex":"F","city":"Eugene","country":"US","country_iso":"us","course":"8k","class":"elite","race":"8K","name":"Alexi Jones”,”_ver":"14","tag":"0000001","bib":"1"}],"info":{"first":"1","last":"1","total":"1","cacheVer":"0~0"}}</p></body></html>

如何防止Nokogiri添加额外的东西？我只是希望它能够准确地返回返回给浏览器的内容。

当我按照另一个SO答案推荐doc = Nokogiri::HTML.fragment(open(url))时，我收到错误：

error: undefined method `strip' for #<StringIO:0x007ff8acb34c30>

Answer 1

Nokogiri假设您已经确定是否收到了适当的解析内容。在将它传递给Nokogiri之前，由您来检查。

不要使用

doc = Nokogiri::HTML(open(url))

您可以查看“CONTENT-TYPE”的返回HTTP标头，对于JSON响应，应该是“application / json”，对于HTML，应该是“TEXT / HTML”。 OpenURI documentation有以下示例：

open("http://www.ruby-lang.org/en") {|f|
  f.each_line {|line| p line}
  p f.base_uri         # <URI::HTTP:0x40e6ef2 URL:http://www.ruby-lang.org/en/>
  p f.content_type     # "text/html"
  p f.charset          # "iso-8859-1"
  p f.content_encoding # []
  p f.last_modified    # Thu Dec 05 02:45:02 UTC 2002
}

或者，您可以查看返回正文的第一个字符，它将告诉您它是HTML / XML还是JSON。前两个将以<开头，JSON将以[或{开头。

这样的事情将是一个不错的开始：

content = open('http://www.example.com').read

if content.lstrip[0] == '<'
  # it's XML/HTML so parse it with Nokogiri
else
  # it's JSON so parse it with the JSON parser
end

如何防止Nokogiri添加不必要的HTML标记？

1 个答案: