无法使用BeautifulSoup从《芝加哥论坛报》上刮取文章的日期

时间:2019-11-01 03:08:48

标签: date web-scraping beautifulsoup python-3.7

我正在网上抓取《芝加哥论坛报》的报纸网站,以获取新闻报道的日期,但我不知道。

import time
import requests
from bs4 import BeautifulSoup
from bs4.element import Tag

url = 'https://www.chicagotribune.com/search/dispatcher.front?page={}&sortby=display_time%20descending&target=stories&spell=on&Query=cybersecurity#trb_search'
pages = 10

for page in range(1, pages+1):
    res = requests.get(url.format(page))
    soup = BeautifulSoup(res.text,"lxml")

    for item in soup.find_all("a", {"class": "trb_search_result_title"}, href=True):
        _href = item.get("href")
        try:
            resp = requests.get(_href)
        except Exception as e:
            try:
                resp = requests.get("https://www.chicagotribune.com"+_href)
            except Exception as e:
                continue

        sauce = BeautifulSoup(resp.text,"lxml")
        timetag = sauce.find('time',{'class': "trb_search_result_datetime"})

        date = None

        if isinstance(timetag, Tag):
            timetag = timetag.get_text().strip()

        print(f'{date}\n')
        time.sleep(3)

为什么我没有得到任何结果。

谢谢。

0 个答案:

没有答案