Question

我想在执行javascript“点击”事件后抓取网页

网络如下所示，

function initPage() {

initCorpInfo();

var Tree = Ext.tree;

var treeRoot = new Tree.TreeNode({
    text: "total",
    id: "root",
    href: "javascript: viewDoc('20150515001896', '4671059', null, null, null, 'dart3.xsd')"
});

    treeNode2 = new Tree.TreeNode({
        text: "4. financial statement",
        id: "17",
        cls: "text",
        listeners: {
            click: function() {viewDoc('20150515001896', '4671059', '17', '1015699', '132786',


    });
}

function viewDoc(rcpNo, dcmNo, eleId, offset, length, dtd) {

currentDocValues.rcpNo = rcpNo;
currentDocValues.dcmNo = dcmNo;
currentDocValues.eleId = eleId;
currentDocValues.offset = offset;
currentDocValues.length = length;
currentDocValues.dtd = dtd;
var params = "";
params += "?rcpNo=" + rcpNo;
params += "&dcmNo=" + dcmNo;
if (eleId != null)
    params += "&eleId=" + eleId;
if (offset != null)
    params += "&offset=" + offset;
if (length != null)
    params += "&length=" + length;
params += "&dtd=" + dtd;
document.getElementById("ifrm").src = "/report/viewer.do" + params;

}

视图源：http://dart.fss.or.kr/dsaf001/main.do?rcpNo=20150515001896 （点击左栏中的4.재무제<）

我可以执行“click：function（）{viewDoc（'20150515001896'，'4671059'，'17'，'1015699'，'132786'，”使用selenium和beautifulsoup？

我应该使用scrapy而不是Beautifulsoup来接近javascript的功能吗？

Answer 1

它只是通过这种方式解决了。

from selenium import webdriver
from bs4 import BeautifulSoup
import time

browser=webdriver.Firefox()
browser.get("http://dart.fss.or.kr/dsaf001/main.do?rcpNo=20150515001896")

soup=BeautifulSoup(browser.page_source) 
browser.execute_script("viewDoc('20150515001896', '4671059', '17', '1015699', '132786', 'dart3.xsd');")

使用selenium和Beautifulsoup执行javascript后抓取网页

1 个答案: