Question

我正在使用下面的脚本从网站上抓取航班数据

import requests
import bs4
import csv

root_url = 'http://www.flightradar24.com/data/flights/3k601/'

response = requests.get(root_url)
soup = bs4.BeautifulSoup(response.text)


try:
    table = soup.find('table')
    rows = table.find_all('tr')
    heads = [i.text.strip() for i in table.select('thead th')]
    for tr in table.select('tbody tr'):
        with open('flight_data.csv', 'a', newline='') as csvfile:
            fieldnames = ['flight', 'From', 'To', 'Date', 'Aircraft', 'STD', 'ATD', 'STA', 'Status'] 
            writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

        flight_data = {}
        flight_data['flight'] = tr['data-flight-number']
        flight_data['From'] = " ".join(tr.select('td')[1].get_text().split())
        flight_data['To'] = " ".join(tr.select('td')[2].get_text().split())
        flight_data['Date'] = tr['data-date']
        flight_data['Aircraft'] = " ".join(tr.select('td')[3].get_text().split())
        flight_data['STD'] = tr.select('td')[4].get_text()
        flight_data['ATD'] = tr.select('td')[5].get_text()
        flight_data['STA'] = tr.select('td')[6].get_text()
        flight_data['Status'] = " ".join(tr.select('td')[7].get_text().split())
        print (flight_data)
        writer.writerow(flight_data)

except AttributeError as e:
    raise ValueError("No valid table found")

如果表格中有数据，这是有效的，就像flight pages like this我试图对密钥进行条件检查一样

if tr['data-flight-number'] is None:

但是tr['data-flight-number']不存在，那么在我继续废弃数据之前，如何测试此表是否为空？

Answer 1

使用get

if tr.get('somekey', None) is None

使用Python进行抓取时的KeyError

1 个答案: