我正在使用 [cqlsh 5.0.1 | Cassandra 2.2.1 | CQL规范3.3.0 |原生协议v4] 版本。我有2个节点cassandra集群,复制因子为2。
$ nodetool status test_keyspace
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 10.xxx.4.xxx 85.32 GB 256 100.0% xxxx-xx-xx-xx-xx rack1
UN 10.xxx.4.xxx 80.99 GB 256 100.0% x-xx-xx-xx-xx rack1
[我已将数字替换为x]
这是键空间定义。
cqlsh> describe test_keyspace;
CREATE KEYSPACE test_keyspace WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '2'} AND durable_writes = true;
CREATE TABLE test_keyspace.test_table (
id text PRIMARY KEY,
listids map<int, timestamp>
) WITH bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX list_index ON test_keyspace.test_table (keys(listids));
id
是唯一的,listids's
密钥的基数接近1000.我在此密钥空间中有数百万条记录。
我想获取具有特定键的记录数以及这些记录的列表。我从cqlsh:
尝试了这个查询select count(1) from test_table where listids contains key 12;
几秒钟后出现此错误:
ReadTimeout: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
我已经修改了cqlshrc和cassandra.yaml中的超时参数。
cat /etc/cassandra/conf/cassandra.yaml | grep read_request_timeout_in_ms
#read_request_timeout_in_ms: 5000
read_request_timeout_in_ms: 300000
cat ~/.cassandra/cqlshrc
[connection]
timeout = 36000
request_timeout = 36000
client_timeout = 36000
当我检查/var/log/cassandra/system.log
时,我只得到了这个 -
WARN [SharedPool-Worker-157] 2016-07-25 11:56:22,010 SelectStatement.java:253 - Aggregation query used without partition key
我正在使用Java客户端构建我的代码。 java客户端也获得了很多读取超时。一种解决方案可能是重新构建我的数据,但这需要更多时间(虽然我不确定)。有人可以建议快速解决这个问题吗?
添加统计信息:
$ nodetool cfstats test_keyspace
Keyspace: test_keyspace
Read Count: 5928987886
Read Latency: 3.468279416568199 ms.
Write Count: 1590771056
Write Latency: 0.02020026287239664 ms.
Pending Flushes: 0
Table (index): test_table.list_index
SSTable count: 9
Space used (live): 9664953448
Space used (total): 9664953448
Space used by snapshots (total): 4749
Off heap memory used (total): 1417400
SSTable Compression Ratio: 0.822577888909709
Number of keys (estimate): 108
Memtable cell count: 672265
Memtable data size: 30854168
Memtable off heap memory used: 0
Memtable switch count: 0
Local read count: 1718274
Local read latency: 63.356 ms
Local write count: 1031719451
Local write latency: 0.015 ms
Pending flushes: 0
Bloom filter false positives: 369
Bloom filter false ratio: 0.00060
Bloom filter space used: 592
Bloom filter off heap memory used: 520
Index summary off heap memory used: 144
Compression metadata off heap memory used: 1416736
Compacted partition minimum bytes: 73
Compacted partition maximum bytes: 2874382626
Compacted partition mean bytes: 36905317
Average live cells per slice (last five minutes): 5389.0
Maximum live cells per slice (last five minutes): 51012
Average tombstones per slice (last five minutes): 2.0
Maximum tombstones per slice (last five minutes): 2759
Table: test_table
SSTable count: 559
Space used (live): 62368820540
Space used (total): 62368820540
Space used by snapshots (total): 4794
Off heap memory used (total): 817427277
SSTable Compression Ratio: 0.4856571513639344
Number of keys (estimate): 96692796
Memtable cell count: 2587248
Memtable data size: 27398085
Memtable off heap memory used: 0
Memtable switch count: 558
Local read count: 5927272991
Local read latency: 3.788 ms
Local write count: 559051606
Local write latency: 0.037 ms
Pending flushes: 0
Bloom filter false positives: 4905594
Bloom filter false ratio: 0.00023
Bloom filter space used: 612245816
Bloom filter off heap memory used: 612241344
Index summary off heap memory used: 196239565
Compression metadata off heap memory used: 8946368
Compacted partition minimum bytes: 43
Compacted partition maximum bytes: 1916
Compacted partition mean bytes: 173
Average live cells per slice (last five minutes): 1.0
Maximum live cells per slice (last five minutes): 1
Average tombstones per slice (last five minutes): 1.0
Maximum tombstones per slice (last five minutes): 1
答案 0 :(得分:0)
您可以重新设计表格,也可以在多个较小的查询中拆分查询。
您正在选择使用辅助索引而不使用分区键(这就是警告告诉您的内容)。这样做,您基本上执行全表扫描。您的节点必须查看每个分区才能满足您的请求。
不改变数据模型的解决方案是迭代所有分区并在每个分区运行一次查询。
if (...)
set (ver 1)
else ()
set (ver 2)
endif ()
my_macro${ver} # this is my idea
这样,您的节点就知道您正在寻找这些信息的分区。然后,您必须在客户端聚合这些查询的结果。
答案 1 :(得分:0)
我遇到了同样的问题。试着 1)#也可以设置为None来禁用:home_timeout =在家庭.cassandra中的cqlshrc中为None。没有帮助。
2)增加ym.cassandra.yaml中的超时* timeout_in_ms
也没有帮助。最后,我决定在我的java代码中运行select子句循环并收到计数。 1200万行给了我7秒的计数。它很快。
Cluster cluster = Cluster.builder()
.addContactPoints(serverIp)
.build();
session = cluster.connect(keyspace);
String cqlStatement = "SELECT count(*) FROM imadmin.device_appclass_attributes";
//String cqlStatement = "SELECT * FROM system_schema.keyspaces";
for (Row row : session.execute(cqlStatement)) {
System.out.println(row.toString());
}