我正在使用来自多个站点的nodeJS（TypeScript）构建一个Web爬虫（我是新手，但可以尝试：“ D”来学习。
无论如何，问题出在following code上，当我subscribe和console.log时，返回的value如所假设！什么都没发生！。
console.log中string内的subscribe()硬constructor() {
this.websitesUrls.subscribe(
data => {
this.intialScraping([data[0]]);
}
);
}
都没有显示！
但是它要订阅我的意思是函数可以工作，但是我当然需要数据。
构造函数：
intialScraping(newsPapers: { title: string, href: string }[]) {
console.log('Intializing scrapping');
for (let i = 0; i < newsPapers.length; i++) {
const newsPaper = newsPapers[i];
switch (newsPaper.href) {
case "http://test.com":
console.log('Sracping on: ', newsPaper.title);
let uri = newsPaper.href + '/%D8%A3%D8%AE%D8%A8%D8%A7%D8%B1-%D9%88%D8%AA%D9%82%D8%A7%D8%B1%D9%8A%D8%B1'
RxHR.get(uri)
.pipe(
map(data => {
let $ = cheerio.load(data.body);
const articlesUrl = $('#infinite .item > a').map(function (this: any) {
return $(this).attr('href')
}).get();
return articlesUrl;
}),
switchMap(urls => {
let data = [];
for (let i = 0; i < urls.length; i++) {
const url = urls[i];
// console.log('start looping', url);
let data$ = RxHR.get(url).pipe(
map(data => {
let $ = cheerio.load(data.body);
return {
title: $('h1[itemprop=headline]').text().trim(),
image: $('.article-image img').attr('src'),
content: $('.details').text()
};
})
)
data.push(data$); // if i console here there is an output
} // loop ending
return forkJoin(data);
})
).subscribe( data => {
console.log('Working'); // no-output
console.log(data[0].title); // no-output
})
break;
default:
break;
}
}
}
IntialScraping方法：
console.log在上面的代码中，我在// Demonstrate AWT Checkbox Group
import java.awt.*;
import java.awt.event.*;
public class CBGroup extends Frame implements ItemListener {
String msg = "";
Checkbox windows, android, solaris, mac;
CheckboxGroup cbg;
public CBGroup() {
// Use a flow layout
setLayout (new FlowLayout());
// Create a checkbox group
cbg = new CheckboxGroup();
// Create the checkboxes and include them in the group
windows = new Checkbox("windows", cbg, true);
android = new Checkbox("android", cbg, false);
solaris = new Checkbox("solaris", cbg, false);
mac = new Checkbox("mac", cbg, false);
// Add item listeners
windows.addItemListener(this);
android.addItemListener(this);
solaris.addItemListener(this);
mac.addItemListener(this);
addWindowListener(new WindowAdapter () {
public void windowClosing (WindowEvent we) {
System.exit(0);
}
});
}
public void itemStateChanged (ItemEvent ie) {
repaint();
}
// Display current state of the check boxes
public void paint (Graphics g) {
msg = "Current selection: ";
msg += cbg.getSelectedCheckbox().getLabel();
g.drawString(msg, 20, 120);
}
public static void main(String[] args) {
CBGroup appwin = new CBGroup();
appwin.setSize(new Dimension (240, 180));
appwin.setTitle("CBGroup");
appwin.setVisible(true);
}
}
上发表了评论，不起作用！
注意：如果在“ pipe”“ switchMap”中进行操作，它会为我提供数据！