我如何使用python selenium来提取“: Sahih al-Bukhari 248
”
以下似乎不起作用
reference = find_element_by_xpath(".//div[3]/table/tbody/tr[1]/td[2]").text
print reference
请参阅下面的HTML代码
<div class="actualHadithContainer">
<!-- Begin hadith -->
<a name="1"></a>
<div class="englishcontainer">
<div class="english_hadith_full" style="display: block;">
<div class="hadith_narrated"><p>Narrated `Aisha:</p></div>
<div class="text_details">
<p>Whenever the Prophet (ﷺ) took a bath after Janaba he started by washing his hands and then performed ablution like that for the prayer. After that he would put his fingers in water and move the roots of his hair with them, and then pour three handfuls of water over his head and then pour water all over his body.</p></div>
<div class="clear"></div></div></div>
<div class="arabic_hadith_full arabic"><span class="arabic_sanad arabic"></span>
<span class="arabic_text_details arabic">حَدَّثَنَا عَبْدُ اللَّهِ بْنُ يُوسُفَ، قَالَ أَخْبَرَنَا مَالِ</span><span class="arabic_sanad arabic"></span></div>
<!-- End hadith -->
<div class="bottomItems">
<table class="hadith_reference" cellspacing="0" cellpadding="0">
<tbody><tr><td><b>Reference</b></td>
<td> : Sahih al-Bukhari 248</td></tr>
<tr><td>In-book reference</td>
<td> : Book 5, Hadith 1</td></tr>
<tr><td>USC-MSA web (English) reference</td><td> : Vol. 1, Book 5, Hadith 248</td></tr>
<tr><td> <i>(deprecated numbering scheme)</i></td></tr></tbody></table><div class="hadith_permalink"><a href="javascript: void(0);" onclick="reportHadith(2490, 'h102490')">Report Error</a> | <span class="sharelink" onclick="share('/bukhari/5/1')">Share</span></div></div>
<div class="clear"></div></div>
我正在使用下面的代码提取其他项目,但上述所需的摘录有困难。
代码:
from selenium import webdriver
import os
import re
driver = webdriver.PhantomJS()
driver.implicitly_wait(30)
driver.set_window_size(1120, 550)
driver.get("https://www.sunnah.com/bukhari/5");
print driver.title
print driver.find_element_by_css_selector('.book_page_english_name').text
print driver.find_element_by_xpath('//*[@id="main"]/div[2]/div[1]/div[3]').text
for person in driver.find_elements_by_class_name('actualHadithContainer'):
try:
title1 = person.find_element_by_xpath('.//div[@class="hadith_narrated"]/p').text
if title1:
print title1
else:
print "exception"
title1 = person.find_element_by_xpath('.//div[@class="hadith_narrated"]').text
print title1
title2 = person.find_element_by_xpath('.//div[@class="text_details"]/p').text
if title2:
print title2
else:
title2 = person.find_element_by_xpath('.//div[@class="text_details"]').text
print title2
reference = find_element_by_xpath(".//div[3]/table/tbody/tr[1]/td[2]").text
print reference
except:
print "exception"
答案 0 :(得分:3)
使用selenium API时,您应该执行一些任务,例如单击按钮或滚动到底部。
当您需要从HTML中提取信息时,您应该使用BeautifulSoup,这很简单:
from selenium import webdriver
import os
import re
from bs4 import BeautifulSoup
driver = webdriver.PhantomJS()
driver.implicitly_wait(30)
driver.set_window_size(1120, 550)
driver.get("https://www.sunnah.com/bukhari/5")
soup = BeautifulSoup(driver.page_source, 'lxml')
soup.find(name='table', class_='hadith_reference').tr.text
此页面是静态的,您应该使用请求:
import requests
from bs4 import BeautifulSoup
r = requests.get("https://www.sunnah.com/bukhari/5")
soup = BeautifulSoup(r.text, 'lxml')
for div in soup.find_all(class_='actualHadithContainer'):
ref = div.find(name='table', class_='hadith_reference').tr.text
print(ref)
出:
Reference : Sahih al-Bukhari 248
Reference : Sahih al-Bukhari 249
Reference : Sahih al-Bukhari 250
Reference : Sahih al-Bukhari 251
Reference : Sahih al-Bukhari 252
Reference : Sahih al-Bukhari 253
Reference : Sahih al-Bukhari 254