Question

我是Beautifulsoup的新手，并且我一直试图将单个td项加载到MySQL表中。如何分隔每个td以便随后可以填充到MySQL表中？

我试图按td标签分解内容，但在那儿我也做错了。

foo %>%
  map_dfr(c("a", "b", "c"), ~ mutate(., key = .x))

我正在尝试获取以下td类以与以下参数匹配：

td类“ firstCol”-stockName td类“ dataCol”-lastValue td类“ dataCol priceDown-priceChange td类“ dataCol最后的价格下降-percentChange

Answer 1

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script> <p>Select Email templates below</p> <form id="mainForm" name="mainForm"> <input type="radio" name="gender" value="Mr." />Mr. <input type="radio" name="gender" value="Mrs." />Mrs. </form> <br> <textarea id="textarea" placeholder="please input your texts here" oninput="myInput()"/></textarea> Real time generated email template <textarea class="form-control" rows="5" id="result" name="text"></textarea>为您提供行中所有row.find_all('td')的列表，因此您只需转换为包含每个td中的文本的列表

td

然后您有all_td = row.find_all('td') arguments = [x.text.strip() for x in all_td]

的参数

INSERT

显示的数据

import urllib.request
from bs4 import BeautifulSoup

html_url = 'https://markets.wsj.com/'
html_doc = urllib.request.urlopen(html_url).read()

soup = BeautifulSoup(html_doc, 'html.parser')
markets = soup.find(id='majorStockIndexes_moduleId')

marketRows = markets.tbody.find_all('tr')

for row in marketRows:
    all_td = row.find_all('td')
    arguments = [x.text.strip() for x in all_td]
    print(arguments)
    # ... here INSERT data to database ...

Answer 2

此代码将为您提供帮助，您只需要配置MySQL凭据即可。

import urllib
import mysql.connector
from bs4 import BeautifulSoup

html_url = 'https://markets.wsj.com/'
html_doc = urllib.request.urlopen(html_url).read()

soup = BeautifulSoup(html_doc, 'html.parser')
markets = soup.find(id='majorStockIndexes_moduleId')

marketRows = markets.tbody.find_all('tr')


mydb = mysql.connector.connect(
  host="localhost",
  user="root",
  passwd="root",
    database="mydatabase"
)
mycursor = mydb.cursor()


for column in marketRows:
    stockName = column.find('td',class_='firstCol').text.strip()
    lastValue = column.find('td',class_='dataCol').text
    priceChange = column.find('td',class_='priceDown').text
    percentChange = column.find('td',class_='last').text

    print("stockname :",stockName)
    print('lastValue :',lastValue)
    print("priceChange :",priceChange)
    print("percentChange :",percentChange)
    print()

    query = "INSERT INTO MarketData1 (recordID, stock, last, priceChange, percentChange) VALUES (NULL, %s, %s, %s, %s)"

    arguments = (stockName, lastValue, priceChange, percentChange)

    mycursor.execute(query, arguments)

    mydb.commit()

如何将用Beautifulsoup抓取的数据移动到MySQL数据库？

2 个答案: