如何使用python获取谷歌新闻标题和搜索关键词?

时间:2017-01-04 20:05:39

标签: python web-scraping python-requests lxml

我正在开展一个项目来浏览谷歌新闻头条并找到关键词。

我希望它:   - 将标题输入文本文件   - 删除逗号,撇号,引号,标点符号等   - 搜索关键词

这是我到目前为止的代码。我正在成为头条新闻,我现在只需要解析每个标题中的关键字。

from lxml import html
import requests

# Send request to get the web page
response = requests.get('http://news.google.com')

# Check if the request succeeded (response code 200)
if (response.status_code == 200):

# Parse the html from the webpage
pagehtml = html.fromstring(response.text)

# search for news headlines
news = pagehtml.xpath('//h2[@class="esc-lead-article-title"] \
                      /a/span[@class="titletext"]/text()')

# Print each news item in a new line
print("\n".join(news))

1 个答案:

答案 0 :(得分:0)

好吧我修好了。

from lxml import html
import requests

# Send request to get the web page
response = requests.get('https://news.google.com/news/section?cf=all&pz=1&topic=b&siidp=b458d5455b7379bd8193a061024cd11baa97&ict=ln')

# Check if the request succeeded (response code 200)
if (response.status_code == 200):

# Parse the html from the webpage
pagehtml = html.fromstring(response.text)

# search for news headlines
news = pagehtml.xpath('//h2[@class="esc-lead-article-title"] \
                      /a/span[@class="titletext"]/text()')

# Print each news item in a new line
print("\n \n".join(news))

tf = open("headlines.txt", "w")

tf.write("\n \n".join(news).lower())

tf.close()
# puts as lower case in text file named headlines

with open('headlines.txt', 'r') as inF:
    for line in inF:
        if 'inflation' in line:
             print "\n" + "    " + line
# searches for 'inflation' (or whatever query) and prints in indented on a new line