Question

我在json txt文件中从twitter中提取了4gb。现在我正在尝试浏览文件并提取用户位置。当我运行下面的脚本时，我得到错误：

文件＆＃34; filepath / test.py＆＃34;，第18行，in 如果推特[＆＃39;用户＆＃39;] [＆＃39; id＆＃39;]： KeyError：＆＃39;用户＆＃39;

收集的推文中是否可能缺少用户ID？我以为它不能为空。我收集了较小的样本，在4个中有3个我得到相同的错误，它仅适用于我没有发现任何json结构差异的一个数据集

    import json

# Tweets are stored in in file "fname". In the file used for this script,
# each tweet was stored on one line
fname = 'test_with_sample.json'
with open(fname, 'r') as f:
    # Create dictionary to later be stored as JSON. All data will be included
    # in the list 'data'
    users_with_geodata = {
        "data": []
    }
    all_users = []
    total_tweets = 0
    geo_tweets = 0
    for line in f:
        tweet = json.loads(line)

        if tweet['user']['id']:
            total_tweets += 1
            user_id = tweet['user']['id']
            if user_id not in all_users:
                all_users.append(user_id)

                # Give users some data to find them by. User_id listed separately
                # to make iterating this data later easier
                user_data = {
                    "user_id": tweet['user']['id'],
                    "features": {
                        "name": tweet['user']['name'],
                        "id": tweet['user']['id'],
                        "screen_name": tweet['user']['screen_name'],
                        "tweets": 1,
                        "location": tweet['user']['location'],
                    }
                }

                if tweet['place']:
                    user_data["features"]["primary_geo"] = tweet['place']['full_name'] + ", " + tweet['place'][
                        'country']
                    user_data["features"]["geo_type"] = "Tweet place"
                else:
                    user_data["features"]["primary_geo"] = tweet['user']['location']
                    user_data["features"]["geo_type"] = "User location"
                # Add only tweets with some geo data to .json. Comment this if you want to include all tweets.
                if user_data["features"]["primary_geo"]:
                    users_with_geodata['data'].append(user_data)
                    geo_tweets += 1

            # If user already listed, increase their tweet count
            elif user_id in all_users:
                for user in users_with_geodata["data"]:
                    if user_id == user["user_id"]:
                        user["features"]["tweets"] += 1
    #except KeyError:
    #    pass

    # Count the total amount of tweets for those users that had geodata
    for user in users_with_geodata["data"]:
        geo_tweets = geo_tweets + user["features"]["tweets"]
    # Get some aggregated numbers on the data
    print
    "The file included " + str(len(all_users)) + " unique users who tweeted with or without geo data"
    print
    "The file included " + str(
        len(users_with_geodata['data'])) + " unique users who tweeted with geo data, including 'location'"
    print
    "The users with geo data tweeted " + str(geo_tweets) + " out of the total " + str(total_tweets) + " of tweets."
# Save data to JSON file
with open('users_geo_sample.json', 'w') as fout:
    fout.write(json.dumps(users_with_geodata, indent=4))

Answer 1

为`if tweet [＆＃39; user＆＃39;] [＆＃39; id＆＃39;]＆＃39;添加例外处理是错误的，继续循环：

try:
      ...code..
except KeyError:
  continue

python3.4 twitter数据抓错误：KeyError：＆＃39; user＆＃39;

1 个答案: