使用Subprocess避免长时间运行的任务断开discord.py bot的连接?

时间:2018-12-03 03:25:07

标签: python-3.x subprocess python-asyncio discord.py

我为Discord服务器创建了一个机器人,该机器人针对给定的subreddit进入Reddit API,并根据您输入的subreddit在Discord聊天中发布当天的前10名结果。它不理会自己的帖子,实际上只发布图片和GIF。 Discord消息命令如下所示:=get funny awww news programming,当每个子Reddit从Reddit API(PRAW)获取结果时,将其发布。这项工作没有问题。我知道该漫游器可以访问API并发布到不和谐的功能。

我添加了另一个命令=getshuffled,该命令将所有来自subreddit的结果放入一个较大的列表中,然后在发布之前将其洗牌。最多约50个subreddits的要求,这确实非常有效。

这是我需要的帮助:

由于结果列表如此之大,从100多个subreddits中获得了1000多个结果,因此该机器人在遇到非常大的请求时崩溃了。根据我从问题yesterday得到的帮助,我了解出了什么问题。该机器人正在启动,正在与我的Discord服务器通信,当我向它传递一个长请求时,它在Reddit API调用完成时停止与服务器通信太长时间,并且Discord连接失败。

所以,我我需要做的是为代码创建一个子进程,该子进程进入Reddit API并提取结果(我认为这会使不和谐的连接保持运行状态) ,然后在完成后将这些结果传回bot。...

或者...这是Asyncio可以自己处理的事情...

正如我所知道的,我在处理子流程时遇到了困难。

基本上,我或者需要这个子过程技巧的帮助,或者需要知道我是否是白痴,并且Asyncio可以为我处理所有这一切。我认为这只是“我不知道我所不知道的”实例之一。

因此,回顾一下:该机器人运行良好,只对少量的subreddit进行了改组。它通过发送的args(subreddits),获取每个帖子的信息,然后在发布不和谐的链接之前重新整理。问题在于,当它是约50+个更大的子集合时。为了使它能够使用更大的数量,我需要让Reddit调用NOT阻止主要的不和谐连接,这就是为什么我试图建立一个子进程。

Python版本是3.6,Discord.py版本是0.16.12 该机器人在PythonAnywhere上托管并运行

代码:

from redditBot_auth import reddit

import discord
import asyncio
from discord.ext.commands import Bot
#from discord.ext import commands
import platform
import subprocess
import ast

client = Bot(description="Pulls posts from Reddit", command_prefix="=", pm_help = False)

@client.event
async def on_ready():
    return await client.change_presence(game=discord.Game(name='Getting The Dank Memes')) 

def is_number(s):
    try:
        int(s)
        return True
    except:
        pass

def show_title(s):
    try:
        if s == 'TITLES':
            return True
    except:
        pass

async def main_loop(*args, shuffled=False):
    print(type(args))

    q=10

    #This takes a integer value argument from the input string.
    #It sets the number variable,
    #Then deletes the number from the arguments list.
    title = False
    for item in args:
        if is_number(item):
            q = item
            q = int(q)
            if q > 15:
                q=15
            args = [x for x in args if not is_number(x)]

        if show_title(item):
            title = True
            args = [x for x in args if not show_title(x)]

    number_of_posts = q * len(args)
    results=[]

    TESTING = False #If this is turned to True, the subreddit of each post will be posted. Will use defined list of results


    if shuffled == False: #If they don't want it shuffled

        for item in args:
            #get subreddit results
            #post links into Discord as it gets them
            #The code for this works

    else: #if they do want it shuffled
        output = subprocess.run(["python3.6", "get_reddit.py", "*args"])
        results = ast.literal_eval(output.decode("ascii"))
        # ^^ this is me trying to get the results back from the other process.

。这是我的get_reddit.py文件:

#THIS CODE WORKS, JUST NEED TO CALL THE FUNCTION AND RETURN RESULTS
#TO THE MAIN_LOOP FUNCTION

from redditBot_auth import reddit
import random

def is_number(s):
    try:
        int(s)
        return True
    except:
        pass

def show_title(s):
    try:
        if s == 'TITLES':
            return True
    except:
        pass

async def get_results(*args, shuffled=False):

    q=10

    #This takes a integer value argument from the input string.
    #It sets the number variable,
    #Then deletes the number from the arguments list.
    title = False
    for item in args:
        if is_number(item):
            q = item
            q = int(q)
            if q > 15:
                q=15
            args = [x for x in args if not is_number(x)]

        if show_title(item):
            title = True
            args = [x for x in args if not show_title(x)]

    results=[]

    TESTING = False #If this is turned to True, the subreddit of each post will be posted. Will use defined list of results.
    NoGrabResults = False

    #This pulls the data and creates a list of links for the bot to post

    if NoGrabResults == False:
        for item in args:
            try:
                #get the posts
                #put them in results list    

            except Exception as e:
                #handle error
                pass

        try:
            #print('____SHUFFLED___')
            random.shuffle(results)
            random.shuffle(results)
            random.shuffle(results)

        except:
            #error stuff

        print(results)
#I should be able to read that print statement for the results, 
#and then use that in the main bot function to post the results.

@client.command()
async def get(*args, brief="say '=get' followed by a list of subreddits", description="To get the 10 Top posts from a subreddit, say '=get' followed by a list of subreddits:\n'=get funny news pubg'\n would get the top 10 posts for today for each subreddit and post to the chat."):
    #sr = '+'.join(args)
    await main_loop(*args)

#THIS POSTS THE POSTS RANDOMLY   
@client.command()
async def getshuffled(*args, brief="say '=getshuffled' followed by a list of subreddits", description="Does the same thing as =get, but grabs ALL of the posts and shuffles them, before posting."):

    await main_loop(*args, shuffled=True)


client.run('my ID')

更新:遵循建议,我通过ThreadPoolExecutor传递了命令,如下所示:

async def main(*args, shuffled):

    if shuffled==True:

        with concurrent.futures.ThreadPoolExecutor() as pool:
            results = await asyncio.AbstractEventLoop().run_in_executor(
                executor=pool, func=await main_loop(*args, shuffled=True))
            print('custom thread pool', results)

但是当脚本尝试与Discord对话时,这仍然会导致错误:

ERROR:asyncio:Task was destroyed but it is pending!
task: <Task pending coro=<Client._run_event() running at /home/GageBrk/.local/lib/python3.6/site-packages/discord/client.py:307> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f28acd8db28>()]>>
Event loop is closed
Destination must be Channel, PrivateChannel, User, or Object. Received NoneType
Destination must be Channel, PrivateChannel, User, or Object. Received NoneType
Destination must be Channel, PrivateChannel, User, or Object. Received NoneType
...

它可以正确发送结果,但是不和谐仍然失去连接。

1 个答案:

答案 0 :(得分:3)

praw依赖于requests库,这是同步的,意味着代码正在阻塞。如果阻止代码执行时间太长,这可能会导致您的机器人死机。

要解决此问题,可以创建一个单独的线程来处理阻塞代码。下面是一个示例。请注意,blocking_function将如何使用time.sleep阻止10分钟(600秒)。这应该足以冻结僵尸并最终使其崩溃。但是,由于该函数使用run_in_executor在其自己的线程中,因此该漫游器继续正常运行。

import time
import asyncio
from discord.ext import commands
from concurrent.futures import ThreadPoolExecutor

def blocking_function():
    print('entering blocking function')
    time.sleep(600)
    print('sleep has been completed')
    return 'Pong'

client = commands.Bot(command_prefix='!')

@client.event
async def on_ready():
    print('client ready')

@client.command()
async def ping():
    loop = asyncio.get_event_loop()
    block_return = await loop.run_in_executor(ThreadPoolExecutor(), blocking_function)
    await client.say(block_return)

client.run('token')