多处理django模型数据库

时间:2017-07-28 03:22:29

标签: django database multiprocessing

我的数据库在执行这个脚本时,数据库已经丢失了数据,可能会或多或少,会有很多空数据。谢谢 以下配置信息列表: SQLITE3 Django1.9.8 Python 2.7.6

def flush_price():
    logging.error("Sub-process(es) begin.")
    logging.error(int(time.time()))
    key = '2500_wine_info.xlsx'
    lists = import_excel(key)
    lwin11s = []
    for item in lists:
        lwin11s.append(str(item['LWIN11'])[:11])
    contracts = Contract.objects.filter(
        is_del=False,
        wine__lwin11__in=lwin11s
    )
    lwins = []
    for contract in contracts:
        lwins.append(str(contract.wine.lwin))
    new_arrs = arr_split(lwins, 50)
    now = conversion_reduce_8_time()
    for index in range(1, 3641):
        date_now = (now - datetime.timedelta(days=index)).strftime('%Y-%m-%d')
        pool = multiprocessing.Pool(processes=len(new_arrs))
        for new_arr in new_arrs:
            pool.apply_async(request_price, (new_arr, date_now, ))
        pool.close()
        pool.join()
        break
    logging.error(int(time.time()))
    logging.error("Sub-process(es) done.")


def request_price(new_arr, date_now):
    headers = {
        'CLIENT_KEY': CLIENT_KEY,
        'CLIENT_SECRET': CLIENT_SECRET,
        'ACCEPT': 'application/json',
        'CONTENT-TYPE': 'application/json',
    }
    data = {
        'lwin': new_arr,
        'priceType': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K'],
        "currency": "EUR",
        "priceDate": date_now,
    }
    price_obj = download_page(VINEX_PRICE_API, request_type=1, data=json.dumps(data), headers=headers)
    if price_obj.status_code == 200:
        price_obj = price_obj.json()
        if price_obj['httpCode'] == '200':
            lwin_details = price_obj['lwinDetail']
            for lwin_detail in lwin_details:
                # 取得参数
                lwin = lwin_detail['lwin']
                # 取得对象信息
                contract = Contract.objects.filter(is_del=False, wine__lwin=lwin).first()
                wine = contract.wine
                # 设置历史数据更新
                RedWinePriceData.objects.filter(
                    is_del=False,
                    contract=contract,
                    lwin=lwin,
                ).first()
                redwinepricedata = RedWinePriceData()
                redwinepricedata.contract = contract
                redwinepricedata.lwin = lwin
                # 改变数据
                redwinepricedata.priceDate = format_time(detail['priceDate'])
                redwinepricedata.save()
                redwinepricedata.created = add_conversion_8_time(date_now)
                redwinepricedata.save()

Change the database information before; The changed database information

1 个答案:

答案 0 :(得分:0)

Pool.apply_async也类似于Python的内置应用程序,除了调用立即返回而不是等待结果。返回ApplyResult对象。您调用其get()方法来检索函数调用的结果。 get()方法一直阻塞,直到函数完成。因此,pool.apply(FUNC,  args,kwargs)相当于pool.apply_async(func,  args,kwargs).get()。