Question

我有cron任务的服务器应用程序（在自己的线程中），我想将数据插入mongodb数据库，我想避免死锁或其他多线程问题。

我的代码：

from multiprocessing.dummy import Pool as ThreadPool
from pymongo import MongoClient
import sched
import time

TIME_INTERVAL = 3
THREAD_NUMBER = 4

s = sched.scheduler(time.time, time.sleep)
pool = ThreadPool(THREAD_NUMBER)

websites = [
    "website1",
    "website2",
    "website3",
    "website4",
]

def insert_to_mongo(result):
    #that is proper way ?
    mongo_client = MongoClient('localhost', 27017)
    dic = mongo_client["cjgs"]["tracks"]
    dic.insert({"Result": result})

def parsing_site(station):
    print "Doing stuff for ", station
    insert_to_mongo("Result for " + station)

def recursion(sc, station):
    parsing_site(station)
    sc.enter(TIME_INTERVAL, 1, recursion, (sc, station,))

def run_cron_task(station):
    s.enter(TIME_INTERVAL, 1, recursion, (s, station,))
    s.run()

pool.map(run_cron_task, websites)

在这种情况下如何使用mongodb？如何使用装饰器和其他语法糖以更python的方式编写此代码？

Answer 1

您可以使用装饰器将所有线程内容封装到其中，如下所示：

def your_decorator(fun):
    # your threading stuff with fun

@decorator
def you_fun():
   # etc

但我建议你更好地看一下MongoDB本身推荐的Two Phase Commit。

在多线程应用程序中使用mongodb的正确方法

1 个答案: