CherryPy会话和大型对象?

时间:2015-02-24 20:30:20

标签: python cherrypy

我有一个CherryPy Webapp,我最初使用基于文件的会话编写。我会不时在会话中存储潜在的大对象,例如运行报告的结果 - 我提供了以各种格式下载报告结果的选项,我不想在以下时重新运行查询用户选择下载是因为可能会获得不同的数据。使用基于文件的会话时,这很好。

现在我正在考虑将第二台服务器联机的可能性,因此我需要能够在服务器之间共享会话数据,因为使用memchached会话存储类型是最合适的。我简要介绍了使用PostgreSQL存储类型,但是这个选项记录很少,而且从我能找到的内容中可能会被破坏。所以我实现了memcached选项。

然而,现在,我遇到了一个问题,当我尝试将某些对象保存到会话时,我得到一个“AssertionError:id xxx未设置的会话数据”。我假设这是由于对象大小超过了CherryPy会话后端或memcached中设置的任意限制,但我真的不知道,因为异常没有告诉我为什么没有设置。我已经将memcached中的对象大小限制增加到最大128MB以查看是否有帮助,但它没有 - 而且这可能不是一个安全的选项。

那么我的解决方案是什么?有什么方法可以使用memcached会话存储来存储任意大的对象?我是否需要为这些对象“推出自己的”基于数据库的解决方案?问题可能不是基于尺寸的吗?或者我还缺少另一种选择吗?

3 个答案:

答案 0 :(得分:2)

我使用mysql来处理我的樱桃会话。只要对象是可序列化的(可以被pickle),您就可以将它作为blob(二进制大对象)存储在mysql中。这是您希望用于mysql会话存储的代码......

https://bitbucket-assetroot.s3.amazonaws.com/Lawouach/cherrypy/20111008/936/mysqlsession.py?Signature=gDmkOlAduvIZS4WHM2OVgh1WVuU%3D&Expires=1424822438&AWSAccessKeyId=0EMWEFSGA12Z1HF1TZ82

"""
MySQLdb session module for CherryPy by Ken Kinder <http://kenkinder.com/>

Version 0.3, Released June 24, 2000.

Copyright (c) 2008-2009, Ken Kinder
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

    * Redistributions of source code must retain the above copyright notice,
    this list of conditions and the following disclaimer.

    * Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in the
    documentation and/or other materials provided with the distribution.

    * Neither the name of the Ken Kinder nor the names of its contributors
    may be used to endorse or promote products derived from this software
    without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
import MySQLdb
import cPickle as pickle
import cherrypy
import logging
import threading

__version__ = '0.2'

logger = logging.getLogger('Session')

class MySQLSession(cherrypy.lib.sessions.Session):
    ##
    ## These can be over-ridden by config file
    table_name = 'web_session'
    connect_arguments = {}

    SCHEMA = """create table if not exists %s (
            id varchar(40),
            data text,
            expiration_time timestamp
        ) ENGINE=InnoDB;"""

    _database = None

    def __init__(self, id=None, **kwargs):
        logger.debug('Initializing MySQLSession with %r' % kwargs)
        for k, v in kwargs.items():
            setattr(MySQLSession, k, v)

        self.db = self.get_db()
        self.cursor = self.db.cursor()

        super(MySQLSession, self).__init__(id, **kwargs)

    @classmethod
    def get_db(cls):
        ##
        ## Use thread-local connections
        local = threading.local()
        if hasattr(local, 'db'):
            return local.db
        else:
            logger.debug("Connecting to %r" % cls.connect_arguments)
            db = MySQLdb.connect(**cls.connect_arguments)
            cursor = db.cursor()
            cursor.execute(cls.SCHEMA % cls.table_name)
            db.commit()
            local.db = db

            return db

    def _load(self):
        logger.debug('_load %r' % self)
        # Select session data from table
        self.cursor.execute('select data, expiration_time from %s '
                            'where id = %%s' % MySQLSession.table_name, (self.id,))
        row = self.cursor.fetchone()
        if row:
            (pickled_data, expiration_time) = row
            data = pickle.loads(pickled_data)

            return data, expiration_time
        else:
            return None

    def _save(self, expiration_time):
        logger.debug('_save %r' % self)
        pickled_data = pickle.dumps(self._data)

        self.cursor.execute('select count(*) from %s where id = %%s and expiration_time > now()' % MySQLSession.table_name, (self.id,))
        (count,) = self.cursor.fetchone()
        if count:
            self.cursor.execute('update %s set data = %%s, '
                                'expiration_time = %%s where id = %%s' % MySQLSession.table_name,
                                (pickled_data, expiration_time, self.id))
        else:
            self.cursor.execute('insert into %s (data, expiration_time, id) values (%%s, %%s, %%s)' % MySQLSession.table_name,
                                (pickled_data, expiration_time, self.id))
        self.db.commit()

    def acquire_lock(self):
        logger.debug('acquire_lock %r' % self)
        self.locked = True
        self.cursor.execute('select id from %s where id = %%s for update' % MySQLSession.table_name,
                            (self.id,))
        self.db.commit()

    def release_lock(self):
        logger.debug('release_lock %r' % self)
        self.locked = False
        self.db.commit()

    def clean_up(self):
        logger.debug('clean_up %r' % self)
        self.cursor.execute('delete from %s where expiration_time < now()' % MySQLSession.table_name)
        self.db.commit()

    def _delete(self):
        logger.debug('_delete %r' % self)
        self.cursor.execute('delete from %s where id=%%s' % MySQLSession.table_name, (self.id,))
        self.db.commit()

    def _exists(self):
        # Select session data from table
        self.cursor.execute('select count(*) from %s '
                            'where id = %%s and expiration_time > now()' % MySQLSession.table_name, (self.id,))
        (count,) = self.cursor.fetchone()
        logger.debug('_exists %r (%r)' % (self, bool(count)))
        return bool(count)

    def __del__(self):
        logger.debug('__del__ %r' % self)
        self.db.commit()
        self.db.close()
        self.db = None

    def __repr__(self):
        return '<MySQLSession %r>' % (self.id,)

cherrypy.lib.sessions.MysqlSession = MySQLSession

那么你的webapp.py会是这样的......

 from mysqlsession import MySQLSession 
 import cherrypy 
 import logging 

 logging.basicConfig(level=logging.DEBUG) 

 sessionInfo = { 
     'tools.sessions.on': True, 
     'tools.sessions.storage_type': "Mysql", 
     'tools.sessions.connect_arguments': {'db': 'sessions'}, 
     'tools.sessions.table_name': 'session' 
 } 

 cherrypy.config.update(sessionInfo) 

 class HelloWorld: 
     def index(self): 
         v = cherrypy.session.get('v', 1) 
         cherrypy.session['v'] = v+1 
         return "Hello world! %s" % v 

     index.exposed = True 

 cherrypy.quickstart(HelloWorld()) 

如果你需要把一些物体放在那里做这样的事......

import pickle

    pickledThing = pickle.dumps(YourObject.GetItems(), protocol=0, fix_imports=False)

希望这有帮助!

答案 1 :(得分:1)

听起来你想存储对存储在Memcache中的对象的引用,然后在需要时将其拉回来,而不是依靠状态来处理加载/保存。

答案 2 :(得分:1)

根据您的解释,我可以得出结论,概念上混合用户会话和缓存并不是一个好主意。主要设计的会话是保持用户身份状态。因此它具有安全措施,锁定,避免并发更改等方面。会话存储通常也是不稳定的。因此,如果您的意思是将会话用作缓存,您应该了解会话的真正工作方式以及后果。

我建议您这样做以建立域模型的正常缓存,以生成报告数据并保持会话以进行身份​​识别。

CherryPy详细信息

默认的CherryPy会话实现会锁定会话数据。在OLAP情况下,您的用户可能无法执行并发请求(例如,打开另一个选项卡),直到报告完成。但是,可以选择手动锁定管理。

PostgreSQL会话存储已损坏,可能会在下一版本中删除。

Memcached会话存储不会实现分布式锁定,因此请确保使用一致的规则来平衡服务器上的用户。