我为Discord服务器创建了一个机器人,该机器人针对给定的subreddit进入Reddit API,并根据您输入的subreddit在Discord聊天中发布当天的前10名结果。它不理会自己的帖子,实际上只发布图片和GIF。 Discord消息命令如下所示:=get funny awww news
,当每个子reddit从Reddit API获取结果时,将其发布。这项工作没有问题。我知道该漫游器可以访问API并发布到不和谐的功能。
这是我需要解决的问题:
我添加了另一个命令=getshuffled
,该命令将所有来自subreddit的结果放入一个较大的列表中,然后在发布之前将其洗牌。最多约50个subreddits的要求,这确实非常有效。
由于结果列表如此之大,从100多个subreddits中获得了1000多个结果,因此该机器人在遇到非常大的请求时崩溃了。现在,我想我知道问题出在:该机器人托管在PythonAnywhere上,在那里我只有3GB的RAM。现在,在我的脑海中,即使列表很大,它也不应该占用太多内存。没门。所以,我想我没有清除内存吗?
我敢肯定,从Reddit提取数据并创建大结果列表会引起很多麻烦,然后当它尝试改组并将结果发布到下一个函数中时,它将耗尽内存,并且PythonAnywhere会杀死过程。
这就是我遇到的问题:我不太确定python如何管理函数之间的内存,即使我做对了也不知道。我认为这很简单,例如我如何调用函数或某些东西,或者我不了解Asyncio的工作原理。
这是PythonAnywhere关于RAM的内容:
由于系统RAM是PythonAnywhere服务器上最稀缺的资源之一,因此我们将您的进程的最大内存大小限制为3GB。这是每个进程的限制,而不是系统范围的限制,因此,如果您有更大的内存需求,则可以通过运行多个较小的进程来执行所需的处理。 如果一个进程超过内存限制,它将被杀死。
在这方面的任何帮助将不胜感激!
(此外,如果代码中有任何让您摇头的东西,请告诉我,我确定我犯了一些错误的操作错误!谢谢大家)
编辑:Python版本为3.6,Discord.py版本为0.16.12
编辑2:我检查了服务器日志:这是运行大量结果时抛出的错误(这是在main_post函数上抛出的,我已经注释了该行):
ERROR:asyncio:Task was destroyed but it is pending!
task: <Task pending coro=<Client._run_event() running at /home/GageBrk/.local/lib/python3.6/site-packages/discord/client.py:307> wait_for=<Future pending cb=[BaseSelectorEventLoop._sock_connect_done(13)(), <TaskWakeupMethWrapper object at 0x7f0aa5bbaa38>()]>>
代码:
import logging
import socket
import sys
import random
import os
from redditBot_auth import reddit
import discord
import asyncio
from discord.ext.commands import Bot
#from discord.ext import commands
import platform
@client.event
async def on_ready():
return await client.change_presence(game=discord.Game(name='Getting The Dank Memes'))
def is_number(s):
try:
int(s)
return True
except:
pass
def show_title(s):
try:
if s == 'TITLE':
return True
if s == 'TITLES':
return True
if s == 'title':
return True
if s == 'titles':
return True
except:
pass
#This gets results for each subreddit and puts into nested list with format
#[[title1, URL1], [title2, URL2], [title3, URL3]]
#It then passes that results list to the function that posts to Discord
async def main_loop(*args, shuffled=False):
q=10
#This takes a integer value argument from the input string.
#It sets the number variable,
#Then deletes the number from the arguments list.
#same with the title variable
title = False
for item in args:
if is_number(item):
q = item
q = int(q)
if q > 15:
q=15
args = [x for x in args if not is_number(x)]
if show_title(item):
title = True
args = [x for x in args if not show_title(x)]
number_of_posts = q * len(args)
results=[]
#results = [[] for x in range(number_of_posts)] #create nested lists for each post. This is ALL the links that will be posted.
TESTING = False #If this is turned to True, the subreddit of each post will be posted. Will use defined list of results
NoGrabResults = False
i = 0
await client.say('*Posting ' + str(number_of_posts) + ' posts from ' + str(len(args))+' subreddits*')
#This pulls the data and creates a list of links for the bot to post
if NoGrabResults == False: #This is for testing, ignore
for item in args:
try:
#subreddit_results = [[] for x in range(q)] #create nested lists for each subreddit.
#This allows for duplicate deletion within the subreddit results,
#rather than going over the entire 'results' list.
subreddit_results=[]
e = 0 #counter for the subreddit_results list.
#await client.say('<'+item+'>')
subreddit = reddit.subreddit(item)
Day = subreddit.top('day', limit= q*2)
Week = subreddit.top('week', limit = q*2)
Month = subreddit.top('month', limit = q*2)
Year = subreddit.top('year', limit = q*2)
AllTime = subreddit.top('all', limit = q*2)
print(item)
for submission in Day:
post = []
if len(subreddit_results) < q :
if submission.is_self is False:
if '/v.redd.it/' not in submission.url:
#print(submission.url)
if '.gif' or 'imgur.com' or 'gfycat' in submission.url:
if submission.url not in subreddit_results:
post.append(submission.title)
post.append(submission.url)
#post.append(item)
subreddit_results.append(post)
#print('pulled posts from Daily')
if len(subreddit_results) < q :
#print('getting posts from Weekly')
for submission in Week:
post = []
if len(subreddit_results) < q :
if submission.is_self is False:
if '/v.redd.it/' not in submission.url:
#print(submission.url)
if '.gif' or 'imgur.com' or 'gfycat' in submission.url:
if submission.url not in subreddit_results:
post.append(submission.title)
post.append(submission.url)
#post.append(item)
subreddit_results.append(post)
if len(subreddit_results) < q :
#print('getting posts from Monthly')
for submission in Month:
post = []
if len(subreddit_results) < q :
if submission.is_self is False:
if '/v.redd.it/' not in submission.url:
#print(submission.url)
if '.gif' or 'imgur.com' or 'gfycat' in submission.url:
if submission.url not in subreddit_results:
post.append(submission.title)
post.append(submission.url)
#post.append(item)
subreddit_results.append(post)
if len(subreddit_results) < q :
#print('getting posts from Yearly')
for submission in Year:
post = []
if len(subreddit_results) < q :
if submission.is_self is False:
if '/v.redd.it/' not in submission.url:
if '.gif' or 'imgur.com' or 'gfycat' in submission.url:
if submission.url not in subreddit_results:
post.append(submission.title)
post.append(submission.url)
#post.append(item)
subreddit_results.append(post)
if len(subreddit_results) < q :
#print('getting posts from All Time')
for submission in AllTime:
post = []
if len(subreddit_results) < q :
if submission.is_self is False:
if '/v.redd.it/' not in submission.url:
if '.gif' or 'imgur.com' or 'gfycat' in submission.url:
if submission.url not in subreddit_results:
post.append(submission.title)
post.append(submission.url)
#post.append(item)
subreddit_results.append(post)
#print (subreddit_results)
# If they don't want shuffled results, it will post results
# to Discord as it gets them, instead of creating the nested list
if shuffled == False:
await client.say('<'+item+'>')
for link in subreddit_results:
if title==True:
await client.say(link[0]) #title
await client.say(link[1]) #post url 1
if TESTING == True:
await client.say(link[2]) #subreddit
if title==True:
await client.say("_") #spacer to seperate title from post above
else:
for link in subreddit_results:
results.append(link)
except Exception as e:
if 'Redirect to /subreddits/search' or '404' in str(e):
await client.say('*'+item+' failed...* '+'`Subreddit Does Not Exist`')
if '403' in str(e):
await client.say('*'+item+' failed...* '+'`Access Denied`')
print(str(e) + ' --> ' + item)
pass
print ('results loaded')
await main_post(results, shuffled, title, TESTING)
else:
from Test_args import LargeResults as results
#print(results)
await main_post(results, shuffled, title, TESTING)
。
#this shuffles the posts and posts to Discord.
async def main_post(results, shuffled, title, TESTING):
try:
if shuffled == True:
print('____SHUFFLED___')
random.shuffle(results)
random.shuffle(results)
random.shuffle(results)
#This posts the links in the 'results' list to Discord
for post in results:
try:
# THIS IS WHERE THE PROGRAM IS FAILING!!
if title==True:
await client.say(post[0]) #title
await client.say(post[1]) #post url
if TESTING == True:
await client.say(post[2]) #subreddit
if title==True:
await client.say("_") #spacer to separate title from post above
except Exception as e:
print(e)
pass
await client.say('ALL DONE! ! !')
except Exception as e:
print (e)
pass
await client.say('`' +str(e) +'`')
。
@client.command()
async def get(*args, brief="say '=get' followed by a list of subreddits", description="To get the 10 Top posts from a subreddit, say '=get' followed by a list of subreddits:\n'=get funny news pubg'\n would get the top 10 posts for today for each subreddit and post to the chat."):
#sr = '+'.join(args)
await main_loop(*args)
#THIS POSTS THE POSTS RANDOMLY
@client.command()
async def getshuffled(*args, brief="say '=getshuffled' followed by a list of subreddits", description="Does the same thing as =get, but grabs ALL of the posts and shuffles them, before posting."):
await main_loop(*args, shuffled=True)
client.run('my ID')