Python 3脚本使用xpath获取所有谷歌搜索结果,没有selenium自动化和没有BeautifulSoup

时间:2018-03-28 13:24:43

标签: python web-scraping

我已经尝试了谷歌自定义搜索引擎API但从未给我准确的结果。我在谷歌搜索结果中尝试废弃所有网址但是在几次请求后谷歌阻止我作为机器人。有没有办法获取准确的结果(差不多) 我试过以下代码

import search_google.api
from apiclient.discovery import build

buildargs = {
        "serviceName": "customsearch",
        "version": "v1",
        "developerKey": "dkey"
            }
cseargs = {
       "q": "search",
       "cx": "ckey",
       "num": 8
          }
results = search_google.api.results(buildargs, cseargs)
#results.preview()

# Obtain the url links from the search
# Links are inside results['items'] list
links = results.get_values('items', 'link')

# Obtain the url links from the search
links = results.links
for a in links:
    print (links)

# Save the search result links to a text file
#  results.save_links('links.txt')



# Set developer key and CSE ID
dev_key = 'dkey'

cse_id = 'ckey'




# Obtain search results from Google CSE
service = build("customsearch", "v1", developerKey=dev_key)

results = service.cse().list(q='infotek', cx=cse_id).execute()

#print("\n",results.get_values('items','link'))

#res = service.cse().list(q="keyword query", cx=my_cse_id, 
**kwargs).execute()

0 个答案:

没有答案