使用pycassa将键和行缓存设置为列族?

时间:2014-07-18 14:58:35

标签: python caching cassandra thrift pycassa

我想知道pycassa是否可以将keys_cachedrows_cached字段设置为特定列族(甚至是键空间)?如此处所示使用cassandra cli:http://www.datastax.com/docs/1.0/configuration/storage_configuration

我找到了alter_column_family函数,该函数采用了文档中的key_cache_size参数:http://pycassa.github.io/pycassa/api/pycassa/system_manager.html

但是当我在设置列族(http://pycassa.github.io/pycassa/assorted/pycassa_shell.html)的密钥缓存大小后使用pycassaShell检查缓存值时,它仍然告诉我:

Row Cache:                       None%
Key Cache:                       None%

并且在两者之后也没有性能/内存使用的差异,并且因为alter_column_family需要**kwargs个参数,并且显然没有检查它们,所以使用任何参数名称调用成功...所以我认为key_cache_size对应什么......而且我没有找到可能的可选参数的文档。

收到alter_column_family电话后,这是cassandra日志:

INFO 17:39:26,338 Update ColumnFamily '53c7deadcc9b10271a2df9f0/B' From org.apache.cassandra.config.CFMetaData@3fbd01a[cfId=2dd38542-82ea-381f-be51-44a30af61f24,ksName=53c7deadcc9b10271a2df9f0,cfName=B,cfType=Standard,comparator=org.apache.cassandra.db.marshal.IntegerType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.DoubleType,keyValidator=org.apache.cassandra.db.marshal.IntegerType,minCompactionThreshold=4,maxCompactionThreshold=32,column_metadata={java.nio.HeapByteBuffer[pos=0 lim=3 cap=3]=ColumnDefinition{name=6b6579, validator=org.apache.cassandra.db.marshal.IntegerType, type=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null}, java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=76616c7565, validator=org.apache.cassandra.db.marshal.DoubleType, type=COMPACT_VALUE, componentIndex=null, indexName=null, indexType=null}, java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=636f6c756d6e31, validator=org.apache.cassandra.db.marshal.IntegerType, type=CLUSTERING_KEY, componentIndex=null, indexName=null, indexType=null}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=<null>,memtable_flush_period_in_ms=0,caching=KEYS_ONLY,defaultTimeToLive=0,speculative_retry=NONE,indexInterval=128,populateIoCacheOnFlush=false,droppedColumns={},triggers={}] To org.apache.cassandra.config.CFMetaData@11131f6f[cfId=2dd38542-82ea-381f-be51-44a30af61f24,ksName=53c7deadcc9b10271a2df9f0,cfName=B,cfType=Standard,comparator=org.apache.cassandra.db.marshal.IntegerType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.DoubleType,keyValidator=org.apache.cassandra.db.marshal.IntegerType,minCompactionThreshold=4,maxCompactionThreshold=32,column_metadata={java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=636f6c756d6e31, validator=org.apache.cassandra.db.marshal.IntegerType, type=CLUSTERING_KEY, componentIndex=null, indexName=null, indexType=null}, java.nio.HeapByteBuffer[pos=0 lim=3 cap=3]=ColumnDefinition{name=6b6579, validator=org.apache.cassandra.db.marshal.IntegerType, type=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null}, java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=76616c7565, validator=org.apache.cassandra.db.marshal.DoubleType, type=COMPACT_VALUE, componentIndex=null, indexName=null, indexType=null}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=<null>,memtable_flush_period_in_ms=0,caching=KEYS_ONLY,defaultTimeToLive=0,speculative_retry=NONE,indexInterval=128,populateIoCacheOnFlush=false,droppedColumns={},triggers={}]
INFO 17:39:26,349 CFS(Keyspace='system', ColumnFamily='schema_columnfamilies') liveRatio is 5.344978165938865 (just-counted was 5.344978165938865).  calculation took 0ms for 25 cells
INFO 17:39:26,349 Enqueuing flush of Memtable-schema_keyspaces@2042821730(138/8832 serialized/live bytes, 3 ops)
WARN 17:39:26,350 setting live ratio to maximum of 64.0 instead of Infinity
INFO 17:39:26,351 CFS(Keyspace='system', ColumnFamily='schema_keyspaces') liveRatio is 64.0 (just-counted was 64.0).  calculation took 1ms for 0 cells
INFO 17:39:26,351 Writing Memtable-schema_keyspaces@2042821730(138/8832 serialized/live bytes, 3 ops)
INFO 17:39:26,368 Completed flushing var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-jb-79-Data.db (177 bytes) for commitlog position ReplayPosition(segmentId=1405696976943, position=238330)
INFO 17:39:26,373 Enqueuing flush of Memtable-schema_columnfamilies@200691258(1145/6120 serialized/live bytes, 25 ops)
INFO 17:39:26,373 Writing Memtable-schema_columnfamilies@200691258(1145/6120 serialized/live bytes, 25 ops)
INFO 17:39:26,381 Completed flushing var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-jb-74-Data.db (787 bytes) for commitlog position ReplayPosition(segmentId=1405696976943, position=238330)

有一个有趣的caching=KEYS_ONLY参数,它似乎与pycassaShell输出相矛盾,但没有关于此密钥缓存大小的大小,也没有关于行缓存的信息。无论我放key_cache_size=2000000还是None,我都会得到相同的输出。

有人知道如何通过pycassa做到这一点吗?

1 个答案:

答案 0 :(得分:1)

您使用的是什么版本的Cassandra? pycassa在列族属性方面有点过时(特别是在pycassaShell中)。

在Cassandra 1.1的cassandra.yaml中,

key_cache_sizerow_cache_size被全局选项(key_cache_size_in_mbrow_cache_size_in_mb)取代。 caching选项是您应为每列系列设置的选项,可以设置为KEYS_ONLYROWS_ONLYALLNONE

作为旁注,此时你通常应该使用cqlsh而不是cassandra-cli和python CQL driver而不是pycassa(我同时保留两者),特别是对于新项目。