如何在python中同时执行批量请求?

时间:2017-05-17 21:44:17

标签: python multithreading asynchronous praw grequests

我正在使用Reddit api来支持一些subreddits。我想同时upvote多个subreddits,使用praw库与reddit api进行交互。

我想赞成/ r / bookclub的27个“热门”和“上升”帖子:

subreddit = reddit.subreddit("bookclub")

for submission in subreddit.hot(limit=27):
    submission.upvote()

for submission in subreddit.rising(limit=27):
    submission.upvote()

我想使用异步或多处理来更快地完成此操作。我使用grequests库一次完成所有.upvote(),它到目前为止工作,但我不确定它是否真的更好:

subreddit = reddit.subreddit("bookclub")

hot_upvotes = (grequests.get(submission.upvote()) for submission in subreddit.hot(limit=27))
grequests.map(hot_upvotes)

rising_upvotes = (grequests.get(submission.upvote()) for submission in subreddit.rising(limit=27))
grequests.map(rising_upvotes)

两个版本都支持subreddit。我想知道的是,是否可以同时运行hot_upvotesrising_upvotes部分?

1 个答案:

答案 0 :(得分:1)

from threading import Thread
import functools

subreddit = reddit.subreddit("bookclub")

def run_hot_upvotes(subreddit):
    hot_upvotes = (grequests.get(submission.upvote()) for submission in subreddit.hot(limit=27))
    grequests.map(hot_upvotes)

def run_rising_upvotes(subreddit):
    rising_upvotes = (grequests.get(submission.upvote()) for submission in subreddit.rising(limit=27))
    grequests.map(rising_upvotes)

Thread(target = functools.partial(run_rising_upvotes, subreddit)).start()
Thread(target = functools.partial(run_hot_upvotes, subreddit)).start()

尝试以下操作,使两者同时运行,每个都在自己的线程中。