抓取数据时出现IndexError

时间:2019-05-10 23:14:11

标签: python web-scraping

我正在尝试从拍卖目录中抓取一些数据,并在第11行中不断收到“ IndexError:列表索引超出范围”错误。 前3个目录运行良好,但第4个目录出现错误并停止工作。 我试图一步一步下载它,并且一切正常,它们都有这个'auctionId'元素。 可能是什么问题?

auction_url = 'https://bidspirit-portal.global.ssl.fastly.net//services/portal/getAuctionItems?cacheVersion=2019-04-29_18-10-11&cdnSubDomain=ru&intKey={auction_code}'
for i in range (2000, 2010):
    r = requests.get (auction_url.format(auction_code=i))
    parsed_json = r.json()
    auction_id = parsed_json[0]['auctionId']
    print(auction_id, len(auction_id))
    prices = requests.get('https://ru.bidspirit.com/services/account/getAccountActionsForAuction?auctionId=' + str(auction_id))
    parsed_prices = prices.json()
    with open ('input_%s.json' % i, 'w', encoding='utf-8') as f:
        json.dump(parsed_json, f, indent = 2, ensure_ascii=False)
    with open ('prices_%s.json' % i, 'w', encoding='utf-8') as f_1:
        json.dump(parsed_prices, f_1, indent = 2, ensure_ascii=False)

结果我得到

5896c87691c917730e76adb6 24
5896c87691c917730e76adb7 24
589729bc91c917730e76b32d 24
Traceback (most recent call last):
  File "script.py", line 11, in <module>
    auction_id = parsed_json[0]['auctionId']
IndexError: list index out of range

1 个答案:

答案 0 :(得分:0)

有时您的结果为空。所以下面应该做

import requests
import json

auction_url = 'https://bidspirit-portal.global.ssl.fastly.net//services/portal/getAuctionItems?cacheVersion=2019-04-29_18-10-11&cdnSubDomain=ru&intKey={auction_code}'
for i in range (2000, 2010):
    r = requests.get (auction_url.format(auction_code=i))
    parsed_json = r.json()
    if parsed_json:
        auction_id = parsed_json[0]['auctionId']
        print(auction_id, len(auction_id))
        prices = requests.get('https://ru.bidspirit.com/services/account/getAccountActionsForAuction?auctionId=' + str(auction_id))
        parsed_prices = prices.json()
        with open ('input_%s.json' % i, 'w', encoding='utf-8') as f:
            json.dump(parsed_json, f, indent = 2, ensure_ascii=False)
        with open ('prices_%s.json' % i, 'w', encoding='utf-8') as f_1:
            json.dump(parsed_prices, f_1, indent = 2, ensure_ascii=False)