Django减速 - 数千个未使用的数据库调用

时间:2013-05-21 16:03:51

标签: python django performance apache2 psycopg2

我有一个奇怪的问题,主数据库中包含大约27,000个项目的数据库,以及多对多链接这些项目的约400,000个项目。该数据库是一个Postgresql,运行自Psycopg2,位于Apache2上的Mod_wsgi上的Django服务器上。

此特定视图的渲染速度减慢到10-20秒。

我很难找到这种减速的来源。简单的日志记录告诉我实际渲染视图的时间不到75毫秒,因此它不是视图逻辑或模板。并且正在为视图执行的SQL查询总共需要35毫秒。因此,我看不到有几秒钟的操作。

剖析告诉我:

 ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     5597    3.750    0.001    3.820    0.001 /usr/local/lib/python2.7/dist-packages/django/db/backends/postgresql_psycopg2/base.py:52(execute)
303114/66508    2.518    0.000    5.290    0.000 /usr/lib/python2.7/copy.py:145(deepcopy)
   297634    0.762    0.000    0.762    0.000 /usr/lib/python2.7/copy.py:267(_keep_alive)
    16627    0.622    0.000    6.145    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:242(clone)
44159/22079    0.449    0.000    2.531    0.000 /usr/lib/python2.7/copy.py:234(_deepcopy_tuple)
     5597    0.423    0.000    4.653    0.001 /usr/local/lib/python2.7/dist-packages/django/db/backends/util.py:37(execute)
38706/33254    0.374    0.000    2.970    0.000 /usr/local/lib/python2.7/dist-packages/django/utils/tree.py:55(__deepcopy__)
    10961    0.367    0.000    0.599    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/base.py:326(__init__)
    16558    0.322    0.000    8.946    0.001 /usr/local/lib/python2.7/dist-packages/django/db/models/query.py:241(iterator)
     5597    0.293    0.000    2.071    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py:56(as_sql)
     5602    0.266    0.000    0.983    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:1035(add_filter)
     5597    0.232    0.000    0.540    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py:242(get_default_columns)
94039/83135    0.221    0.000    1.783    0.000 /usr/lib/python2.7/copy.py:226(_deepcopy_list)
    44448    0.216    0.000    0.216    0.000 /usr/local/lib/python2.7/dist-packages/django/utils/datastructures.py:121(__init__)
    22209    0.207    0.000    0.207    0.000 /usr/lib/python2.7/encodings/utf_8.py:15(decode)
    22080    0.199    0.000    1.550    0.000 /usr/lib/python2.7/copy.py:306(_reconstruct)
     5723    0.198    0.000   15.201    0.003 /usr/local/lib/python2.7/dist-packages/django/db/models/fields/related.py:367(__get__)
     5597    0.194    0.000    0.323    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:104(__init__)
    16627    0.179    0.000    6.396    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/query.py:904(_clone)
    66702    0.174    0.000    0.174    0.000 /usr/local/lib/python2.7/dist-packages/django/utils/tree.py:18(__init__)
     5597    0.162    0.000    0.787    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py:162(get_columns)
     5597    0.161    0.000    0.302    0.000 /usr/local/lib/python2.7/dist-packages/django/db/backends/postgresql_psycopg2/base.py:160(_cursor)
    44448    0.159    0.000    0.159    0.000 /usr/local/lib/python2.7/dist-packages/django/utils/datastructures.py:116(__new__)
    11182    0.149    0.000    0.304    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py:854()
    16627    0.145    0.000    0.315    0.000 /usr/local/lib/python2.7/dist-packages/django/utils/datastructures.py:136(__deepcopy__)
    40271    0.140    0.000    0.209    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py:40(quote_name_unless_alias)
    16558    0.131    0.000    7.713    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py:763(results_iter)
    34669    0.129    0.000    0.129    0.000 /usr/local/lib/python2.7/dist-packages/django/db/backends/postgresql_psycopg2/operations.py:71(quote_name)
16796/11194    0.128    0.000    0.689    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/where.py:76(as_sql)
    22224    0.123    0.000    0.446    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/query.py:36(__init__)
        1    0.115    0.115    0.115    0.115 /usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py:119(connect)
     5437    0.112    0.000   14.141    0.003 /usr/local/lib/python2.7/dist-packages/django/db/models/query.py:374(get)
     5598    0.111    0.000    1.232    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:1235(add_q)
        1    0.109    0.109    0.109    0.109 /usr/local/lib/python2.7/dist-packages/django/template/loaders/filesystem.py:33(load_template_source)
     5602    0.108    0.000    0.561    0.000 /usr/local/lib/python2.7/dist-packages/django/db/models/sql/where.py:148(make_atom)

所以看起来有成千上万的调用正在进行Base.py和deepcopy,但是当我查看视图的connection.queries值时,总共只有7个,总共大约35毫秒。

之前是否有人遇到此问题?我的视图代码中没有循环,这是正常的启发式,基于pk的查找和保存。

0 个答案:

没有答案