我正在调查我们在生产设置中当前拥有的应用程序每小时收到一次错误。看起来有一个爬虫向我们发送格式不正确的utf8参数。这是我们看到的例外情况：
ArgumentError: invalid byte sequence in UTF-8
这是我们在每次请求时看到的用户代理：
Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)
我们收到Airbrake报告的以下参数：
{
"action": "index",
"controller": "cards",
"format": "",
"utf8": "â?/u201C"
}
回溯：
/usr/local/rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/cgi/util.rb:7 in "gsub"
/usr/local/rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/cgi/util.rb:7 in "escape"
/gems/activesupport-3.2.12/lib/active_support/core_ext/object/to_query.rb:10 in "to_query"
/gems/activesupport-3.2.12/lib/active_support/core_ext/object/to_param.rb:52 in "block in to_param"
/gems/activesupport-3.2.12/lib/active_support/core_ext/object/to_param.rb:51 in "each"
/gems/activesupport-3.2.12/lib/active_support/core_ext/object/to_param.rb:51 in "collect"
/gems/activesupport-3.2.12/lib/active_support/core_ext/object/to_param.rb:51 in "to_param"
/gems/actionpack-3.2.12/lib/action_dispatch/http/url.rb:47 in "url_for"
/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:591 in "url_for"
/gems/actionpack-3.2.12/lib/action_dispatch/routing/url_for.rb:148 in "url_for"
/gems/actionpack-3.2.12/lib/action_controller/caching/actions.rb:172 in "initialize"
/gems/actionpack-3.2.12/lib/action_controller/caching/actions.rb:141 in "new"
/gems/actionpack-3.2.12/lib/action_controller/caching/actions.rb:141 in "filter"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:321 in "around"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:310 in "_callback_around_375"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:214 in "_conditional_callback_around_1195"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:414 in "_run__3684532395506755200__process_action__4186607910431134588__callbacks"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:405 in "__run_callback"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:385 in "_run_process_action_callbacks"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:81 in "run_callbacks"
/gems/actionpack-3.2.12/lib/abstract_controller/callbacks.rb:17 in "process_action"
/gems/actionpack-3.2.12/lib/action_controller/metal/rescue.rb:29 in "process_action"
/gems/actionpack-3.2.12/lib/action_controller/metal/instrumentation.rb:30 in "block in process_action"
/gems/activesupport-3.2.12/lib/active_support/notifications.rb:123 in "block in instrument"
/gems/activesupport-3.2.12/lib/active_support/notifications/instrumenter.rb:20 in "instrument"
/gems/activesupport-3.2.12/lib/active_support/notifications.rb:123 in "instrument"
/gems/actionpack-3.2.12/lib/action_controller/metal/instrumentation.rb:29 in "process_action"
/gems/actionpack-3.2.12/lib/action_controller/metal/params_wrapper.rb:207 in "process_action"
/gems/activerecord-3.2.12/lib/active_record/railties/controller_runtime.rb:18 in "process_action"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:38 in "block in process_action"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:272 in "block in perform_action_with_newrelic_trace"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/agent/method_tracer.rb:235 in "trace_execution_scoped"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:267 in "perform_action_with_newrelic_trace"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:37 in "process_action"
/gems/actionpack-3.2.12/lib/abstract_controller/base.rb:121 in "process"
/gems/actionpack-3.2.12/lib/abstract_controller/rendering.rb:45 in "process"
/gems/actionpack-3.2.12/lib/action_controller/metal.rb:203 in "dispatch"
/gems/actionpack-3.2.12/lib/action_controller/metal/rack_delegation.rb:14 in "dispatch"
/gems/actionpack-3.2.12/lib/action_controller/metal.rb:246 in "block in action"
/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:73 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:73 in "dispatch"
/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:36 in "call"
/gems/journey-1.0.4/lib/journey/router.rb:68 in "block in call"
/gems/journey-1.0.4/lib/journey/router.rb:56 in "each"
/gems/journey-1.0.4/lib/journey/router.rb:56 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:601 in "call"
/gems/rack-pjax-0.7.0/lib/rack/pjax.rb:12 in "call"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/rack/error_collector.rb:12 in "call"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/rack/error_collector.rb:12 in "call"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/rack/agent_hooks.rb:18 in "call"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/rack/browser_monitoring.rb:16 in "call"
/gems/warden-1.2.1/lib/warden/manager.rb:35 in "block in call"
/gems/warden-1.2.1/lib/warden/manager.rb:34 in "catch"
/gems/warden-1.2.1/lib/warden/manager.rb:34 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/best_standards_support.rb:17 in "call"
/gems/rack-1.4.5/lib/rack/etag.rb:23 in "call"
/gems/rack-1.4.5/lib/rack/conditionalget.rb:25 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/head.rb:14 in "call"
/gems/remotipart-1.0.5/lib/remotipart/middleware.rb:30 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/params_parser.rb:21 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/flash.rb:242 in "call"
/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:210 in "context"
/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:205 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/cookies.rb:341 in "call"
/gems/activerecord-3.2.12/lib/active_record/query_cache.rb:64 in "call"
/gems/activerecord-3.2.12/lib/active_record/connection_adapters/abstract/connection_pool.rb:479 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/callbacks.rb:28 in "block in call"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:405 in "_run__4006189938838080721__call__2271109139271149174__callbacks"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:405 in "__run_callback"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:385 in "_run_call_callbacks"
/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:81 in "run_callbacks"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/callbacks.rb:27 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/remote_ip.rb:31 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/debug_exceptions.rb:16 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/show_exceptions.rb:56 in "call"
/gems/railties-3.2.12/lib/rails/rack/logger.rb:32 in "call_app"
/gems/railties-3.2.12/lib/rails/rack/logger.rb:16 in "block in call"
/gems/activesupport-3.2.12/lib/active_support/tagged_logging.rb:22 in "tagged"
/gems/railties-3.2.12/lib/rails/rack/logger.rb:16 in "call"
/gems/actionpack-3.2.12/lib/action_dispatch/middleware/request_id.rb:22 in "call"
/gems/rack-1.4.5/lib/rack/methodoverride.rb:21 in "call"
/gems/rack-1.4.5/lib/rack/runtime.rb:17 in "call"
/gems/activesupport-3.2.12/lib/active_support/cache/strategy/local_cache.rb:72 in "call"
/gems/rack-1.4.5/lib/rack/lock.rb:15 in "call"
/gems/rack-cache-1.2/lib/rack/cache/context.rb:136 in "forward"
/gems/rack-cache-1.2/lib/rack/cache/context.rb:143 in "pass"
/gems/rack-cache-1.2/lib/rack/cache/context.rb:172 in "rescue in lookup"
/gems/rack-cache-1.2/lib/rack/cache/context.rb:168 in "lookup"
/gems/rack-cache-1.2/lib/rack/cache/context.rb:66 in "call!"
/gems/rack-cache-1.2/lib/rack/cache/context.rb:51 in "call"
/gems/utf8-cleaner-0.0.6/lib/utf8-cleaner/middleware.rb:18 in "call"
/gems/railties-3.2.12/lib/rails/engine.rb:479 in "call"
/gems/railties-3.2.12/lib/rails/application.rb:223 in "call"
/gems/railties-3.2.12/lib/rails/railtie/configurable.rb:30 in "method_missing"
/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:552 in "process_client"
/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:632 in "worker_loop"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:22 in "call"
/gems/newrelic_rpm-3.6.0.78/lib/new_relic/agent/instrumentation/unicorn_instrumentation.rb:22 in "block (4 levels) in <top (required)>"
/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:500 in "spawn_missing_workers"
/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:142 in "start"
/gems/unicorn-4.6.2/bin/unicorn:126 in "<top (required)>"
/bin/unicorn:23 in "load"
/bin/unicorn:23 in "<main>"
尽管我努力，但我无法重现这个问题。我想知道是否有其他人之前遇到过这个问题，如果在机架错误之前有什么可以用来清理请求的话。
答案 0 :(得分：1)
有utf8-cleaner gem，但它只处理错误的％编码字符串。它不会清除未编码的utf-8字符串。
所以我分叉了他的gem（https://github.com/lulalala/utf8-cleaner），如果请求url不包含 - ％ - 编码的utf-8字符，则更改语义以响应400错误。您可以再次挖掘我的版本以进行清理，而不是使用400响应。
（但在我看来，已消毒的请求永远不会有用，最好以400回复）
答案 1 :(得分：0)
我设法修复它（在Rails 3.2.18应用程序上），如本主题所述：
https://gist.github.com/joost/ca4eda8f31655cf6095a
只需向Rails应用程序添加一个小中间件文件，它也会返回错误400。