当我尝试提取 amazon.com 的内容时,Node.js请求模块会向控制台提供文本,如下所示:
?y?H>5Z??{???O???↔??????|◄???♦?<∟?h??j??B??43?!>ã?l???∟??? ??M??v6?
eP$r|???$;??☼?Thc???ea??l?p??k?▼??☺↓i???L?v7?x?6??M#tA??↕v?Z)?p1´?vQ??9?ET?1???J
?_c?☼↨u?Î%5↨??q??¶▬l??→1↨?$??h?_??J-;??r???+?▲?F?Hw?♦????lE?Qs?Hx??o9@??V3??
L?bk?fcb??????o?E9??]????"??}x♦7?7r→?z0KE??Z?▬?4?I?A??R↨???/s<???☻V`?f!????3?2;?
?????L???????!?OA9↕iC?/????r?0??U?♫M?♂?}y???=,e?M?↔Q[¶`xn?|B??w?D♫f?↓?↨☻n¶????
zH??4p??☻???O?☻♠????w↨?????P'???z?etXN'?U??`??Z??♀">♀j????????????5???!?????#u??
0X?i?zb?☺?[?&∟?>??‼??Q??+???}???z▲A???9§????O????♠???? ?∟?es???j??D0J?s?[?;U??!
???l0???u i35_???∟x???2<RF???{???\d♥<?8?W?p>◄→]?????¶+???|(???☻z??♦??v??8⌂
,?"▲??∟?l???1?A?7zt??Q,?'??♥?n???♦,??r?N?H\?-?YA>)?♦??|X?C;I?q⌂]r↓?H??¶D??????>C
?X??? ?b???o?_+R?9??8??^??_???‼????_*v?↓?♣ ??"♠?♀!J1?Ib????u??Bg?a?S??↕?d1??&hZ
?H?↑?N♣???!?⌂|b?.0?&'▬?→?C*5ukp?▲4?☻>?7♣??,????2?\??$?X??4?T7???H7?$5?"?????,I?→
h??zy↕?▼???☻7??J]Ab1|rF?&^?↔??J]SG??<??►4?☺?↕?♥B?~P? 9∟?e|.BR?0♥? ???]
相反,当我尝试从 amazon.co.uk 提取数据时,正如我们期望的那样,它会给出如下的html结构化输出:
<html><head>...</head><body>...</body></html>
如何解决第一种情况?如何获取html内容?有什么办法吗?
代码:
const rp = require('request-promise');
const url = 'http://www.amazon.co.uk/gp/product/B0085EY4MS';
const fs = require('fs');
rp(url)
.then(function(html){
fs.writeFile('mynewfile3.txt', html, function (err) {
if (err) throw err;
console.log('Saved!');
});
})
.catch(function(err){
});
答案 0 :(得分:0)
好吧,问题就出在请求的编码上。您可以为编码类型添加一个选项,这样可以解决您的问题。更新后的代码如下:
const rp = require('request-promise');
const fs = require('fs');
const opts = {
uri: "http://www.amazon.com/Fallout-76-PlayStation-4/dp/B07DD9571S",
headers: {
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36",
},
gzip: true, //Added to fix the issue
}
rp(opts)
.then(function(html){
fs.writeFile('mynewfile3.txt', html, function (err) {
if (err) throw err;
console.log('Saved!');
});
})
.catch(function(err){
console.log(err);
});
标题不是必需的。