Question

我正在尝试使django视图生成多个异步http请求并返回组合数据。

以下是一个在单独文件中运行的示例：

import gevent
from gevent import monkey
monkey.patch_all()
import requests


results = {}
processes = []
def fill_results(url, position):
    print(url, position, 'starting')
    results[position] = {'url': url, 'response': requests.get(url)}
    print(url, position, 'ready')


urls = ['http://www.accessgenealogy.com'] * 5


for position, url in enumerate(urls):
    processes.append(gevent.spawn(fill_results, url, position))

gevent.joinall(processes)
print(results)

我得到的输出是：

('http://www.accessgenealogy.com', 0, 'starting')
('http://www.accessgenealogy.com', 1, 'starting')
('http://www.accessgenealogy.com', 2, 'starting')
('http://www.accessgenealogy.com', 3, 'starting')
('http://www.accessgenealogy.com', 4, 'starting')
('http://www.accessgenealogy.com', 3, 'ready')
('http://www.accessgenealogy.com', 0, 'ready')
('http://www.accessgenealogy.com', 4, 'ready')
('http://www.accessgenealogy.com', 2, 'ready')
('http://www.accessgenealogy.com', 1, 'ready')
{0: {'url': 'http://www.accessgenealogy.com', 'response': <Response [200]>}, 1: {'url': 'http://www.accessgenealogy.com', 'response': <Response [200]>}, 2: {'url': 'http://www.accessgenealogy.com', 'response': <Response [200]>}, 3: {'url': 'http://www.accessgenealogy.com', 'response': <Response [200]>}, 4: {'url': 'http://www.accessgenealogy.com', 'response': <Response [200]>}}

因此请求确实是不同步的。

当我将相同的代码粘贴到django 1.9视图中时，monkey.patch_all()打破了django：

Traceback (most recent call last):
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/utils/autoreload.py", line 226, in wrapper
    fn(*args, **kwargs)
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
    self.check_migrations()
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 163, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/db/migrations/executor.py", line 20, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/db/migrations/loader.py", line 49, in __init__
    self.build_graph()
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/db/migrations/loader.py", line 176, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations
    self.ensure_schema()
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 52, in ensure_schema
    if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in cursor
    self.validate_thread_sharing()
  File "/Users/1111/.virtualenvs/django_cpa/lib/python2.7/site-packages/django/db/backends/base/base.py", line 523, in validate_thread_sharing
    % (self.alias, self._thread_ident, thread.get_ident()))
DatabaseError: DatabaseWrapper objects created in a thread can only be used in that same thread. The object with alias 'default' was created in thread id 140735166436112 and this is thread id 4488713392.
<Greenlet at 0x10b8c54b0: wrapper(use_static_handler=True, settings=None, pythonpath=None, verbosity=1, traceback=False, addrport='8000', no_color=False, use_ipv6=False, use_threading=True, use_reloader=True, insecure_serving=False)> failed with DatabaseError

我应该在不破坏django的情况下进行补丁以使其正常工作？

Answer 1

您可以尝试以下几点：

确保您不在greenlets中执行任何数据库操作
在视图中导入gevent和patch，然后在完成后取消。

以下是一个例子：

class MyView(View):

    def get(self, request, *args, **kwargs):
        def fill_results(url, position):
            # Point 1: no DB operations inside here!
            results[position] = {'url': url, 'response': request.get(url)}

        urls = ['http://www.accessgenealogy.com'] * 5

        # import gevent and patch
        import gevent
        from gevent import monkey
        monkey.patch_all()

        processes = []
        for position, url in enumerate(urls):
            processes.append(gevent.spawn(fill_results, url, position))

        gevent.joinall(processes)

        # Point 2: unpatch after use
        reload(socket)

        return HttpResponse()

django中的猴子修补请求

1 个答案: