将LXML与Html,Requests和ETree一起使用,它可以提供链接,但不会让我搜索特定文本的链接

时间:2019-03-05 13:25:27

标签: python web-scraping lxml

我正在尝试从下面提供的链接中提取特定数据。当我运行代码时,它会按预期提供所有href链接,但是当我尝试对同一字符串进行进一步测试但使用contains语法时,它返回为空。

我已经阅读了文档以及DevHints,在我所看到的所有地方,都推荐使用“包含”语法来捕获Im所寻找的内容,因为我所知道的只是将要包含的语法,而不是在哪里或

我正在尝试制造一种刮板以帮助最近被解雇的很多人找到新工作,因此,我们将不胜感激。

代码:

sapply(split(dat, dat$x), function(i) sum(i$a) / sum(i$b))
#        1         2         3 
#0.3448112 0.7289661 0.5581262

1 个答案:

答案 0 :(得分:1)

BeautifulSoup解决方案

from bs4 import BeautifulSoup
import requests

page = requests.get('https://ea.gr8people.com/index.gp?method=cappportal.showPortalSearch&sysLayoutID=123').content

soup = BeautifulSoup(page, 'html.parser')
links = soup.find_all('a')
links = [a for a in links if a.attrs.get('href') and 'opportunityid' in a.attrs.get('href')]
print('-- opportunities --')
for idx, link in enumerate(links):
    print('{}) {}'.format(idx, link))

输出

-- opportunities --
0) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=154761&amp;opportunityid=154761">
                                        2D Capture Artist - 6 month contract
                                    </a>
1) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=154426&amp;opportunityid=154426">
                                        Accounting Supervisor
                                    </a>
2) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=152147&amp;opportunityid=152147">
                                        Advanced Analyst
                                    </a>
3) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=153395&amp;opportunityid=153395">
                                        Advanced UX Researcher
                                    </a>
4) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=151309&amp;opportunityid=151309">
                                        AI Engineer
                                    </a>
5) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=150468&amp;opportunityid=150468">
                                        AI Scientist
                                    </a>
6) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=151310&amp;opportunityid=151310">
                                        AI Scientist - NLP Focus
                                    </a>
7) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=153351&amp;opportunityid=153351">
                                        AI Software Engineer (Apex Legends)
                                    </a>
8) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=152737&amp;opportunityid=152737">
                                        AI Software Engineer (Frostbite)
                                    </a>
9) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=154764&amp;opportunityid=154764">
                                        Analyste Qualité Sénior / Senior Quality Analyst
                                    </a>
10) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=153948&amp;opportunityid=153948">
                                        Animator 1
                                    </a>
11) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=151353&amp;opportunityid=151353">
                                        Applications Agreement Analyst
                                    </a>
12) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=154668&amp;opportunityid=154668">
                                        AR Analyst I
                                    </a>
13) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=153609&amp;opportunityid=153609">
                                        AR Specialist
                                    </a>
14) <a href="index.gp?method=cappportal.showJob&amp;layoutid=2092&amp;inp1541=&amp;inp1375=154773&amp;opportunityid=154773">
                                        Artiste Audio / Audio Artist
                                    </a>