从Twitter获得10,000个用户的追随者总数

时间:2017-09-20 03:18:47

标签: python json twitter

我试图获得每个Twitter用户的关注者数量,以获得通过随机数生成器随机选择的10,000个用户的列表。我正在使用Twitter的GET用户/查找API来执行此操作。由于15个请求/ 15分钟的限制,我无法直接使用GET粉丝。我设法做到这一点,但我的数据不是json格式,我花了整整一天试图让jq处理它(由于信封内的特殊字符而失败)。任何人都可以建议我该怎么做?

我对Twitter的流API没有任何问题,因为数据是JSON格式,jq可以解析。我需要用jq,python和amp; shell脚本。

我的代码获得此回应:(Python)

#!/usr/bin/python

import json
import oauth2 as oauth
from random import randint
import sys
import time

ckey= '<insert-key>'
csecret= '<insert-key>'
atoken= '<insert-key>'
asecret= '<insert-key>'

consumer = oauth.Consumer(key= ckey, secret = csecret)
access_token = oauth.Token(key=atoken, secret=asecret)
client= oauth.Client(consumer, access_token)

# Print to a file
sys.stdout = open('TwitterREST.txt', 'w')

for j in range(0,20):
    for i in range(0, 900):
        user_id =(randint(2361391150, 2361416150))

        f="https://api.twitter.com/1.1/users/lookup.json?user_id="
        s=str(user_id)
        timeline_endpoint = f+s

        response,data=client.request(timeline_endpoint)
        tweets= json.loads(data)

        for tweet in tweets:
            print (data)

    # Twitter only allows 900 requests/15 mins
    time.sleep(900)

TWITTER返回的样本响应:(无法打印出来)

b&#39; [{&#34; id&#34;:2361393867,&#34; id_str&#34;:&#34; 2361393867&#34;,&#34; name&#34;:&#34 ; graam a7bab&#34;,&#34; screen_name&#34;:&#34; bedoo691&#34;,&#34; location&#34;:&#34;&#34;,&#34; description&#34 ;:&#34; \ u0627 \ u0633 \ u062a \ u062a \ u0621 \ u0627 \ u0644 \ u0644 \ u0644 \ u0648 \ u0627 \ u062a \ u0648 \ u0628 \ u0624 \ u0648 \ u0647 \ u0647 \ u0647 ...!* &#34;,&#34; url&#34;:null,&#34; entity&#34;:{&#34; description&#34;:{&#34; urls&#34;:[]}}, &#34;受保护&#34;:false,&#34; followers_count&#34;:1,&#34; friends_count&#34;:6,&#34; listed_count&#34;:0,&#34; created_at&# 34;:&#34; Sun Feb 23 19:03:21 +0000 2014&#34;,&#34; favourites_count&#34;:1,&#34; utc_offset&#34;:null,&#34; time_zone&# 34;:null,&#34; geo_enabled&#34;:false,&#34;已验证&#34;:false,&#34; statuses_count&#34;:7,&#34; lang&#34;:&#34 ; ar&#34;,&#34; status&#34;:{&#34; created_at&#34;:&#34; Tue Mar 04 16:07:44 +0000 2014&#34;,&#34; id&# 34;:440881284383256576, &#34; id_str&#34;:&#34; 440881284383256576&#34;,&#34; text&#34;:&#34; @ Naif8989&#34;,&#34;截断&#34;:false,&# 34;实体&#34;:{&#34; hashtags&#34;:[],&#34; symbols&#34;:[],&#34; user_mentions&#34;:[{&#34; screen_name&#34 ;:&#34; Naif8989&#34;,&#34; name&#34;:&#34; \ u200f naif alharbi&#34;,&#34; id&#34;:540343286,&#34; id_str&#34 ;:&#34; 540343286&#34;,&#34; indices&#34;:[0,9]}],&#34; urls&#34;:[]},&#34; source&#34;: &#34; \ u003ca href = \&#34; http:\ / \ / twitter.com \ / download \ / android \&#34; rel = \&#34; nofollow \&#34; \ u003eTwitter for Android \ u003c \ / a \ u003e&#34;,&#34; in_reply_to_status_id&#34;:437675858485321728,&#34; in_reply_to_status_id_str&#34;:&# 34; 437675858485321728&#34;,&#34; in_reply_to_user_id&#34;:2361393867,&#34; in_reply_to_user_id_str&#34;:&#34; 2361393867&#34;,&#34; in_reply_to_screen_name&#34;:&#34; bedoo691& #34;,&#34; geo&#34;:null,&#34; coordinates&#34;:null,&#34; place&#34;:null,&#34; contributors&#34;:null,&# 34; is_quote_status&#34;:false,&#34; retweet_count&#34;:0,&#34; favorite_count&#34;:0,&#34; favited&#34;:false,&#34;转发&#34; :false,lang&#34;:&#34; und&#34;},contributors_enabled&#34;:false,is_translator&#34;:false,is_translation_enabled&#34;:false,profile_background_color&#34;:&#34; C0DEED&# 34;,profile_background_image_url&#34;:&#34; http:\ / \ / abs.twimg.com \ / images \ / themes \ / theme1 \ /bg.png" ;,亲file_background_image_url_https&#34;:&#34; https:\ / \ / abs.twimg.com \ / images \ / themes \ / theme1 \ /bg.png" ;, profile_background_tile&#34;:false,profile_image_url&#34;: &#34; http:\ / \ / pbs.twimg.com \ / profile_images \ / 437664693373911040 \ /ydODsIeh_normal.jpeg" ;, profile_image_url_https&#34;:&#34; https:\ / \ / pbs.twimg.com \ / profile_images \ / 43764693373911040 \ /ydODsIeh_normal.jpeg" ;, profile_link_color&#34;:&#34; 1DA1F2&#34;,profile_sidebar_border_color&#34;:&#34; C0DEED&#34;,profile_sidebar_fill_color&#34;:&# 34; DDEEF6&#34;,profile_text_color&#34;:&#34; 333333&#34;,profile_use_background_image&#34;:true,has_extended_profile&#34;:false,default_profile&#34;:true,default_profile_image&#34;:false,关注& #34;:false,follow_request_sent&#34;:false,&#34; notifications&#34;:false,&#34; translator_type&#34;:&#34; none&#34; }]&#39;

0 个答案:

没有答案