如何克服＆＃34;未知的编码名称 - macintosh＆＃34;与Nokogiri的错误

时间：2016-10-11 13:51:15

标签： ruby-on-rails ruby encoding nokogiri pre

我正在使用Rails 4.2.7。我目前正在使用以下逻辑来解析Nokogiri的文档：

content.xpath("//pre[@class='text-results']").xpath('text()').to_s

在我的HTML文档中，此内容显示在我的“文本结果”块中：

<pre class="text-results"><html xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:w="urn:schemas-microsoft-com:office:word"
xmlns="http://www.w3.org/TR/REC-html40">

<head>
<meta name=Title content="&lt;p&gt;&lt;a href=http://mychiptime">
<meta name=Keywords content="">
<meta http-equiv=Content-Type content="text/html; charset=macintosh”>…

我包含此部分，因为我的解析因以下错误而消失：

Error during processing: unknown encoding name - macintosh
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node.rb:627:in `find'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node.rb:627:in `serialize'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node.rb:786:in `to_format'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node.rb:642:in `to_html'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node.rb:512:in `to_s'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node_set.rb:187:in `block in each'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node_set.rb:186:in `upto'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node_set.rb:186:in `each'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node_set.rb:218:in `map'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.8.1/lib/nokogiri/xml/node_set.rb:218:in `to_s'
/Users/davea/Documents/workspace/myproject/app/services/onlinerr_service.rb:8:in `pre_process_data'
/Users/davea/Documents/workspace/myproject/app/services/abstract_import_service.rb:77:in `process_my_object_data'
/Users/davea/Documents/workspace/myproject/app/services/onlinerr_my_object_finder_service.rb:82:in `process_my_object_link'
/Users/davea/Documents/workspace/myproject/app/services/abstract_my_object_finder_service.rb:29:in `block in process_data'
/Users/davea/Documents/workspace/myproject/app/services/abstract_my_object_finder_service.rb:28:in `each'
/Users/davea/Documents/workspace/myproject/app/services/abstract_my_object_finder_service.rb:28:in `process_data'
/Users/davea/Documents/workspace/myproject/app/services/run_crawlers_service.rb:18:in `block in run_all_crawlers'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:46:in `each'
/Users/davea/.rvm/gems/ruby-2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:46:in `each'
/Users/davea/Documents/workspace/myproject/app/services/run_crawlers_service.rb:5:in `run_all_crawlers'

有没有办法让Nokogiri忽略这个未知的编码？我正在尝试将<pre>标记内的内容作为文本，因此我不需要对其进行进一步解析。

我在Mac El Capitan上。根据评论，这是我的语言环境设置：

davea$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

2 个答案:

答案 0 :(得分：2)

您的HTML无效。您在<pre>之外有一个<body>标记，因此，Nokogiri必须进行修正，这通常会导致可疑结果。

这就是Nokogiri对该文件所说的话：

doc.errors # => [#<Nokogiri::XML::SyntaxError: htmlParseStartTag: misplaced <html> tag>, #<Nokogiri::XML::SyntaxError: htmlParseStartTag: misplaced <head> tag>, #<Nokogiri::XML::SyntaxError: AttValue: " expected>, #<Nokogiri::XML::SyntaxError: Couldn't find end of Start Tag meta>]
doc.to_html # => "<pre class=\"text-results\">\n\n\n<meta name=\"Title\" content=\"&lt;p&gt;&lt;a href=http://mychiptime\">\n<meta name=\"Keywords\" content=\"\">\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=macintosh”&gt;\n&lt;/head&gt;\n\"></pre>"

仅查看相关行，它也让Nokogiri感到困惑：

doc = Nokogiri::HTML::DocumentFragment.parse('<meta http-equiv=Content-Type content="text/html; charset=macintosh”>')

doc.errors # => [#<Nokogiri::XML::SyntaxError: AttValue: " expected>, #<Nokogiri::XML::SyntaxError: Couldn't find end of Start Tag meta>]
doc.to_html # => "<meta http-equiv=\"Content-Type\" content=\"text/html; charset=macintosh”&gt;\">"

请注意，Nokogiri不会将结束的卷曲引用识别为字符串content="text/html; charset=macintosh”的终结符。

你无法在Nokogiri内解决这个问题。您需要提供适当的结构，并且需要在解析文档之前进行搜索和替换以转换引号。希望文档不会在<body>文本中包含它们，或者您正在修改可能对您有用的文本。

你在他们不应该存在的地方卷曲引用的事实很奇怪。如果您的编辑器正在从直引号转换为引号，那么您需要立即关闭该功能，因为它会对编码造成严重破坏。良好的编码文本编辑器甚至不会因为它们引起的问题而使用卷曲引号。

  

Nokogiri正在抱怨＆＃34; macintosh＆＃34;据我所知，序列。

require 'nokogiri'

doc = Nokogiri::HTML::DocumentFragment.parse('<meta http-equiv=Content-Type content="text/html; charset=macintosh">')
doc.at('meta')['content'] # => "text/html; charset=macintosh"

如果HTML是干净的，它就不在乎。

答案 1 :(得分：0)

See Nokogiri, open-uri, and Unicode Characters

When Nokogiri parses a document, it uses the encoding that the document specifies (unless you explicitly tell it what encoding to use).

"macintosh" is not a default Ruby encoding (see Encoding.list for a list of all encodings Ruby knows).

You can force Nokogiri to use an explicit encoding by passing it as an argument to parse.

# encoding is guessed from the document
doc = Nokogiri::HTML.parse(File.open('test.html'))
doc.xpath("//pre[@class='text-results']").xpath('text()').to_s
ArgumentError: unknown encoding name - macintosh

# force Nokogiri to parse the document as 'utf-8'
doc = Nokogiri::HTML.parse(File.open('test.html'), nil, 'utf-8')
doc.xpath("//pre[@class='text-results']").xpath('text()').to_s
 => "\n\n\n"

The caveat is that Nokogiri really will parse the content as 'utf-8', meaning if any special characters are encoded using some other encoding (like macintosh), they may become garbled.

相关问题
最新问题