将Python2.6转换为Google App Engine兼容(多处理)

时间:2011-05-16 18:25:31

标签: python google-app-engine multiprocessing

我在Python中创建了一个使用多处理的小脚本。我曾想过将它作为一个cron-job在Google App Engine上运行,但遗憾的是Google App Engine不支持多处理。任何人都可以帮助我将其转换为Google App Engine兼容代码(可能使用Google App Engine任务吗?)?

from multiprocessing import Pool
import MySQLdb
import urllib;
import urllib2;

def f(email_url):
    url = "http://my-domain.com/cron.php"
    values = { "email" : email_url[0], "url" : email_url[1] }
    data = urllib.urlencode(values)
    req = urllib2.Request(url, data)
    urllib2.urlopen(req)

if __name__ == '__main__':
    p = Pool()
    emails_urls = list()
    conn = MySQLdb.connect(host = "XXX.XXX.XXX.XXX", user = "USERNAME",
                           passwd = "PASSWORD", db = "MY-DATABASE")
    cursor = conn.cursor()
    cursor.execute ("SELECT email, url FROM data")
    rows = cursor.fetchall()
    for row in rows:
        emails_urls.append((row[0], row[1]))
    cursor.close()
    conn.close()

    p.map(f, emails_urls)

1 个答案:

答案 0 :(得分:5)

查看任务队列。

ca可以将一定量的工作插入到任务队列(=> Thread)中,并设置队列中同时执行的作业数。

看看这里:http://code.google.com/intl/de-DE/appengine/docs/python/taskqueue/