' TB'在分配之前引用

时间:2017-02-11 22:56:06

标签: python multithreading multiprocessing

我在项目中使用from multiprocessing.dummy import Pool。问题是,经过多次成功尝试(获得价格)后,它返回UnboundLocalError: local variable 'tb' referenced before assignment

问题是tb变量在被引用时应该100%分配。如果存在例外情况,则会分配tb,否则会返回变量d,并且不会引用tb

我无法弄清问题可能出在哪里。我将向您展示我的功能 - 您可以看到变量tb,即traceback,必须在其引用的所有情况下分配:

def scan_partner(d):
    success = False
    try:
        from parser import parse_price
        resp = get_price(d.get('url'), d.get('xpath'))
        amount = resp.get('price')

    except XPathEvalError as xee:
        tb = traceback.format_exc()
        ticket_code = 'xpath'
        msg = u'XPath Error'
    except (requests.Timeout, requests.ConnectTimeout, requests.ReadTimeout):
        tb = traceback.format_exc()
        ticket_code = 'timeout'
        msg = u'Timeout'
    except requests.HTTPError:
        tb = traceback.format_exc()
        ticket_code = 'http'
        msg = u'HTTP Error'
    except decimal.DecimalException:
        tb = traceback.format_exc()
        ticket_code = 'decimal'
        msg = u'Decimal Error'
    except requests.ConnectionError:
        tb = traceback.format_exc()
        ticket_code = 'connection'
        msg = u'Connection Error'
    except Exception as e:
        tb = traceback.format_exc()
        ticket_code = 'unknown'
        msg = u'Unknown Error'
    else:
        success = True
        result = {'result': {'success': True,
                             'amount': str(amount)}}
        d.update(result)

    finally:
        if success:
            return d
        d.update({'ticket': {'tb': tb,
                             'msg': msg,
                             'ticket_code': ticket_code},
                  'result': {'success': False}})
        return d


def scan_partners_dummy_pool(dicts_list):
    repeated_results = []

    results_to_repeat = []
    for i in range(settings.ENGINE_NUMBER_OF_SCAN_REPEATS):
        pool = Pool(300)
        results = pool.map(scan_partner, dicts_list)
        pool.close()
        pool.join()
        results_to_go = []
        results_to_repeat = []
        for result in results:
            if result['result']['success']:
                results_to_go.append(result)
            else:
                if result['ticket']['ticket_code']=='timeout':
                    results_to_repeat.append(result)
                else:
                    results_to_go.append(result)
        repeated_results.extend(results_to_go)
        dicts_list = results_to_repeat
    repeated_results.extend(results_to_repeat)

    return repeated_results

追溯:

[2017-02-11 23:33:54,470: ERROR/MainProcess] Task engineapp.tasks.scan_every_20_minutes[91c08f5d-36ad-42bc-805f-8a35c01127e6] raised unexpected: UnboundLocalError("local variable 'tb' referenced before assignment",)
Traceback (most recent call last):
  File "c:\users\milano\pycharmprojects\dropboxworkspaces\priceist_workspace\priceist_venv\lib\site-packages\celery\app\trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "c:\users\milano\pycharmprojects\dropboxworkspaces\priceist_workspace\priceist_venv\lib\site-packages\celery\app\trace.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "C:\Users\Milano\Desktop\Projekty\FS Dropbox\Dropbox\priceist\engineapp\tasks.py", line 18, in scan_every_20_minutes
    scan_all_active_users(20)
  File "c:\users\milano\pycharmprojects\dropboxworkspaces\priceist_workspace\priceist_venv\lib\site-packages\celery\local.py", line 167, in <lambda>
    __call__ = lambda x, *a, **kw: x._get_current_object()(*a, **kw)
  File "c:\users\milano\pycharmprojects\dropboxworkspaces\priceist_workspace\priceist_venv\lib\site-packages\celery\app\trace.py", line 439, in __protected_call__
    return orig(self, *args, **kwargs)
  File "c:\users\milano\pycharmprojects\dropboxworkspaces\priceist_workspace\priceist_venv\lib\site-packages\celery\app\task.py", line 420, in __call__
    return self.run(*args, **kwargs)
  File "C:\Users\Milano\Desktop\Projekty\FS Dropbox\Dropbox\priceist\engineapp\tasks.py", line 132, in scan_all_active_users
    report = scan_user(user.id, simple_schedule)
  File "C:\Users\Milano\Desktop\Projekty\FS Dropbox\Dropbox\priceist\engineapp\tasks.py", line 123, in scan_user
    results = scan_partners_dummy_pool(partners_dicts)
  File "C:\Users\Milano\Desktop\Projekty\FS Dropbox\Dropbox\priceist\engineapp\engine\scraper.py", line 126, in scan_partners_dummy_pool
    if result['ticket']['ticket_code']=='timeout':
  File "c:\python27\Lib\multiprocessing\pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "c:\python27\Lib\multiprocessing\pool.py", line 567, in get
    raise self._value
UnboundLocalError: local variable 'tb' referenced before assignment

PS: 该过程是Celery定期任务。

1 个答案:

答案 0 :(得分:1)

您没有捕获所有可能的异常,因此如果您获得的异常不是Exception的子类,那么except阻止和else阻止都不会执行,但是finally块仍然会被执行。

我会在return中添加else语句,并将finally块的内容移至主体。然后你可能会看到BaseException引起麻烦的任何事情。