Question

因此，我正在尝试从Twitter抓取数据。我设法根据特定的主题标签检索推文，然后使用以下代码将它们转储为JSON格式的文本文件：

import json
import tweepy
import pandas as pd
import time

with open('consumer_key.txt', 'r') as f:
    consumer_key =  f.read()
f.closed

with open('consumer_secret.txt', 'r') as f:
    consumer_secret = f.read()
f.closed

with open('access_key.txt', 'r') as f:
    access_key = f.read()
f.closed

with open('access_secret.txt', 'r') as f:
    access_secret = f.read()
f.closed

#Authentication
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)

api = tweepy.API(auth, wait_on_rate_limit=True,
             wait_on_rate_limit_notify=True) # Connect to the api

if (not api):
    print ("Error")
    sys.exit(-1)

public_tweets = api.home_timeline() # Get the post in the timeline
for tweet in public_tweets:
    print(tweet.text) 

with open('tweets.txt', 'a') as f:
    for tweet in tweepy.Cursor(api.search, q='#apple', count=2, lang='en', since='2017-04-03').items():
        json.dump(tweet._json, f, indent=4)

但是，当我尝试使用

读取文件时

with open('tweets.txt', 'r') as f:
    for line in f:
        tweet = json.loads(line)
        tweet_text = tweet['text']
        print(tweet_text)

它给我一个错误，提示“ JSONDecodeError：期望属性名称用双引号引起：第2行第1列（字符2）”

这是我的文本文件的一部分：

{
    "created_at": "Wed Dec 05 10:37:07 +0000 2018",
    "id": 1070265807492530176,
    "id_str": "1070265807492530176",
    "text": "RT @evankirstel: The Apple Watch Series 3 Made Up Majority of 
Estimated 4.2 Million Q3 2018 Apple Watch Sales #applewatch #apple  
@mactrast\u2026",
    "truncated": false,
    "entities": {
        "hashtags": [
            {
                "text": "applewatch",
                "indices": [
                    110,
                    121
                ]
            },
            {
                "text": "apple",
                "indices": [
                    122,
                    128
                ]
            }
        ],
        "symbols": [],
        "user_mentions": [
            {
                "screen_name": "evankirstel",
                "name": "Evan Kirstel",
                "id": 35203319,
                "id_str": "35203319",
                "indices": [
                    3,
                    15
                ]
            },
            {
                "screen_name": "MacTrast",
                "name": "MacTrast",
                "id": 16711478,
                "id_str": "16711478",
                "indices": [
                    130,
                    139
                ]
            }
        ],

我该如何解决？最终，我要做的是根据键提取数据并将其保存在pandas数据框中。

Answer 1

您需要逐行阅读整个内容，并且不要忘记添加strict=False以允许/将\u2026转换为…

with open('tweets.txt', 'r') as f:
    tweet = json.load(f, strict=False)
    tweet_text = tweet['text']
    print(tweet_text)

抓取Twitter数据时出现JSONDecodeError

1 个答案: