我正在尝试提取中等数据并将其保存到数据库中。我能够将数据提取到列表中,并使用zip函数对其进行汇总。但是当我插入数据时,它并没有被插入。
我已经迭代到zip函数以将数据保存到数据库。但是没有得到所需的结果。我已经成功检索了中等职位,并且数据库中的表顺序没有问题。
import mysql.connector
mydb = mysql.connector.connect(host="localhost", user="root",
password="Jai@1998", database="crawldb")
cursor = mydb.cursor()
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = webdriver.ChromeOptions()
opts.set_headless()
assert opts.headless
browser = webdriver.Chrome(options=opts)
browser.get('https://medium.com/')
comp = browser.find_elements_by_xpath('//div[@class = "streamItem
streamItem-extremePostPreview js-streamItem"]')
authors = []
timeline = []
for i in range(len(comp)):
x = comp[i].find_element_by_xpath('.//div[@class = "postMetaInline
postMetaInline-authorLockup u-padding0 u-flexShrink1 u-
noWrapWithEllipsis"]')
authors.append(x)
y = comp[i].find_element_by_xpath('.//div[@class = "ui-caption u-
noWrapWithEllipsis"] ')
timeline.append(y)
topics = browser.find_elements_by_xpath('//h2[@class = "ui-h2 ui-xs-h4
ui-clamp3"]')
descriptions = browser.find_elements_by_xpath('//div[@class = "ui-
summary ui-clamp2 u-marginTop2"]')
for title, desc, author, time in zip(topics, descriptions, authors,
timeline):
print("topics : descriptions : authors : timeline")
print(title.text + ": " + desc.text + ": "+ author.text + ": " +
time.text, '\n')
sql = "INSERT INTO `DATAS`(topics, descriptions, authors, time)
VALUES(%s , %s , %s , %s)", (title.text, desc.text ,
author.text , time.text)
try:
cursor.executemany(sql)
mydb.commit()
print("data inserted!!")
except:
mydb.rollback()
mydb.close()
print("data not inserted!!")
输出是提取数据和未插入的数据!在SQL中。