我一直在使用scrapoxy抓取一些限制限制的网址。每当它在一段时间后阻止URL时,它就不会在响应头中返回x-cache-proxyname,这样我就无法停止该特定实例并创建另一个实例。以下是完整的标题响应。
{"content-type"=>["text/html"],
"content-length"=>["110778"],
"connection"=>["close"],
"date"=>["Sun, 27 May 2018 18:20:37 GMT"],
"last-modified"=>["Wed, 12 Apr 2017 09:12:05 GMT"],
"etag"=>["\"c465e20f1a2c5681318061d5321e2368\""],
"accept-ranges"=>["bytes"],
"server"=>["AmazonS3"],
"x-cache"=>["Error from cloudfront"],
"via"=>["1.1 7f43afdd7e6d9ba0ebc0701aab572252.cloudfront.net (CloudFront)"],
"x-amz-cf-id"=>["dYWVQe6v1cb7JLQy0WnJE9KP0mtZGyrumbNdORIyaOo7MeaOExpf2w=="]}