从网站获取完整的href属性列表

时间:2017-04-18 12:56:44

标签: javascript jquery html web-scraping

我想从链接结构如下的网站获取为YouTube播放列表生成的所有URLs的完整列表:

<table class="table" id="ListVideo">
    <tbody>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
        .
        . and so on 
        .
        .
    </tbody>
</table>

我可以使用

a tag获取个人href
console.log($("#ListVideo td div a").attr("href"))

给了我链接 https://redirector.googlevideo.com/videoplayback...0Brief%20Overview%20Of%20Android我想要 但是当我尝试使用以下功能循环遍历所有链接时

$("#ListVideo td div a").each(()=> {console.log($(this).attr("href"))})

相反,我得到了未定义。 以下是JSfiddle链接,其中包含我想要获取链接的完整页面源。 有人可以告诉我哪里出错了。感谢

4 个答案:

答案 0 :(得分:2)

使用正确的方法来获得正确的$(this) - 所以在这种情况下不是箭头函数。

var list = [];
$("#ListVideo td div a").each(function() {
  list.push($(this).attr("href"))
});
console.log(list);

使用箭头,您无需使用{/ 1>},如

中所示

Using jQuery $(this) with ES6 Arrow Functions (lexical this binding)

this

https://jsfiddle.net/mplungjan/yr39tmqn/

答案 1 :(得分:2)

首先,如何根据属性选择元素,例如$([href])

其次,()=> {console.log($(this).attr("href"))}日志未定义,因为箭头功能具有不同的this。简而言之,箭头函数不会创建自己的this上下文,因此this在封闭上下文中具有其原始含义。 您可以阅读更多相关信息here。使用经典的function会对你有用。

答案 2 :(得分:1)

这是您的updated fiddle。您可以看到完全注销到控制台的链接数量。

console.log($("#ListVideo td div a").attr("href"));
var hrefArray = [];
$("#ListVideo td div a").each(function(){
    console.log($(this).attr("href"));
  hrefArray.push($(this).attr("href"));
});

console.log(hrefArray.length);

答案 3 :(得分:0)

以下是工作示例,纯javascript代码。我们使用querySelectorAll查找a类中的所有.table个nchors。

&#13;
&#13;
var anchors = document.querySelectorAll('.table a');


// pure js
for (i = 0; i < anchors.length; i++) {
  var anchor = anchors[i];
  
  // You can access href attribute via anchor.href
  console.log(anchor.href);
}

// using forEach, we must use Object.keys 
Object.keys(anchors).forEach(function(key) {
    // console.log(anchors[key].href);  // href
});


// using map, we must use Object.keys 
var listOfAnchors = Object.keys(anchors).map(function(key) {
  return anchors[key].href;
});

// Array of anchors inside our table
// console.log(listOfAnchors);
&#13;
<table class="table" id="ListVideo">
    <tbody>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
        <tr>
            <th scope="row">1</th>
            <td><img style="height: 55px;width: 90px;" src="http://img.youtube.com/vi/mHpvT1P_OAM/default.jpg"></td>
            <td>Android Development for Beginner: Brief Overview Of Android</td>
            <td>
                <div id="Download-mHpvT1P_OAM" link="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"><a download="" href="https://redirector.googlevideo.com/videoplayback?api=youtubemultidownloader.com&amp;mn=sn-p5qlsnz6&amp;signature=B2C601E05D43FADDDDDC595B25F3A77E4CF24FB9.96E1AB91173820178049E0402A3F48E29086AEBB&amp;mime=video%2Fmp4&amp;mt=1492516586&amp;dur=182.415&amp;id=o-AIjyKLM2gc2g7w9RB0izMEVqvaTbIxAJ9iTa0tQakpAb&amp;key=yt6&amp;ip=66.249.83.29&amp;mm=31&amp;mv=m&amp;pl=28&amp;ipbits=0&amp;ms=au&amp;sparams=dur%2Cei%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&amp;expire=1492538256&amp;lmt=1486569580323751&amp;ratebypass=yes&amp;source=youtube&amp;upn=P2VOmq_LKgk&amp;itag=22&amp;requiressl=yes&amp;ei=MP_1WLKeBuKF8gTh35WgDQ&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android&amp;title=01.Android%20Development%20for%20Beginner%3A%20Brief%20Overview%20Of%20Android"
                    class="btn btn-default" type="button">MP4 720P</a> </div>
            </td>
        </tr>
    </tbody>
</table>
&#13;
&#13;
&#13;