open-uri失败,并显示Zlib :: DataError:不正确的数据检查

时间:2019-04-08 15:48:01

标签: ruby zlib

我有一个需要在我的ruby应用程序中下载的图像URL列表(> 1000)。我为此使用open-uri。我知道它有缺陷,但是如果您必须支持http,https和ftp,这很方便。

通话本身就这么简单

file = open(url) # an https url

此后,文件由回形针处理。

问题是,即使一次又一次地使用相同的URL,调用也会不时以Zlib::DataError失败。有时可以下载它们,有时图像会引发错误。这是堆栈跟踪:

/usr/local/lib/ruby/2.6.0/net/http/response.rb:382 in inflate
/usr/local/lib/ruby/2.6.0/net/http/response.rb:382 in block in inflate_adapter
/usr/local/lib/ruby/2.6.0/net/protocol.rb:497 in call_block
/usr/local/lib/ruby/2.6.0/net/protocol.rb:488 in <<
/usr/local/lib/ruby/2.6.0/net/protocol.rb:163 in read
/usr/local/lib/ruby/2.6.0/net/http/response.rb:405 in read
/usr/local/lib/ruby/2.6.0/net/http/response.rb:293 in block in read_body_0
/usr/local/lib/ruby/2.6.0/net/http/response.rb:264 in inflater
/usr/local/lib/ruby/2.6.0/net/http/response.rb:283 in read_body_0
/usr/local/lib/ruby/2.6.0/net/http/response.rb:204 in read_body
/usr/local/lib/ruby/2.6.0/open-uri.rb:352 in block (2 levels) in open_http
/usr/local/lib/ruby/2.6.0/net/http.rb:1518 in block in transport_request
/usr/local/lib/ruby/2.6.0/net/http/response.rb:165 in reading_body
/usr/local/lib/ruby/2.6.0/net/http.rb:1517 in transport_request
/usr/local/lib/ruby/2.6.0/net/http.rb:1479 in request
/gems/newrelic_rpm-5.6.0.349/lib/new_relic/agent/instrumentation/net.rb:36 in block in request_with_newrelic_trace
/gems/newrelic_rpm-5.6.0.349/lib/new_relic/agent.rb:501 in disable_all_tracing
/gems/newrelic_rpm-5.6.0.349/lib/new_relic/agent/instrumentation/net.rb:35 in request_with_newrelic_trace
/gems/elastic-apm-2.3.1/lib/elastic_apm/spies/net_http.rb:54 in block in request
/gems/elastic-apm-2.3.1/lib/elastic_apm.rb:276 in with_span
/gems/elastic-apm-2.3.1/lib/elastic_apm/spies/net_http.rb:51 in request
/usr/local/lib/ruby/2.6.0/open-uri.rb:343 in block in open_http
/usr/local/lib/ruby/2.6.0/net/http.rb:920 in start
/usr/local/lib/ruby/2.6.0/open-uri.rb:337 in open_http
/usr/local/lib/ruby/2.6.0/open-uri.rb:756 in buffer_open
/usr/local/lib/ruby/2.6.0/open-uri.rb:226 in block in open_loop
/usr/local/lib/ruby/2.6.0/open-uri.rb:224 in catch
/usr/local/lib/ruby/2.6.0/open-uri.rb:224 in open_loop
/usr/local/lib/ruby/2.6.0/open-uri.rb:165 in open_uri
/usr/local/lib/ruby/2.6.0/open-uri.rb:736 in open
/usr/local/lib/ruby/2.6.0/open-uri.rb:35 in open

我还用wget下载了图像,以查看服务器是否发送了损坏的图像,但是很难调试:您不确定所有是否正常,因为服务器未在其中返回错误。这样运行,或者服务器一切正常,并且没有错误。

所以我不确定错误是由服务器,open-uri还是...引起的?

  

因此,为了更好地调试此问题,我想了解为什么ruby http这样做inflate,并且如果我可以解决它,或者如果有人以前经历过类似的事情。

0 个答案:

没有答案