我有一个3x节点集群(在相同的16核心盒子上,通过lxc在虚拟盒子中,但是它上面的3TB磁盘上的每个节点都有)。
我的表是这样的:
CREATE TABLE history (
id text,
idx bigint,
data bigint,
PRIMARY KEY (id, idx)
) WITH CLUSTERING ORDER BY (idx DESC)
id将存储一个id为字符串的id,idx是以ms为单位的时间,数据是我的数据。根据我发现的所有示例,这似乎是时间序列数据的正确模式。
我的查询是:
选择idx,来自历史的数据,其中id =?限制2
这将返回最近的2个(基于idx)行。
由于id是分区键而idx是聚类键,因此我发现docs声称这对cassandra非常有效。但我的基准却说不然。
我已经填充了总共400GB(分成3个节点),现在我正在运行来自第二个盒子的查询。使用16或32个线程,我正在运行上述查询,但在3个不同磁盘上运行的3个节点的性能非常低:
throughput: 61 avg time: 614,808 μs
throughput: 57 avg time: 519,651 μs
throughput: 52 avg time: 569,245 μs
所以,每秒约55次查询,每次查询需要半秒钟(有时它们需要200毫秒)
我觉得这个很低。
有人可以告诉我我的架构是否正确以及是否建议架构?如果我的架构是正确的,我怎么能找到出错的地方?
16核盒子上的磁盘IO:
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 0.00 0.00 0.00 0 0
sdb 135.00 6.76 0.00 6 0
sdc 149.00 6.99 0.00 6 0
sdd 124.00 7.21 0.00 7 0
cassandras不会使用超过1个cpu核心。
编辑:通过跟踪,当我为1个id运行一个简单的查询时,我会收到如下所示的很多行:
Key cache hit for sstable 33259 | 20:16:26,699 | 127.0.0.1 | 5830
Seeking to partition beginning in data file | 20:16:26,699 | 127.0.0.1 | 5833
Bloom filter allows skipping sstable 33256 | 20:16:26,699 | 127.0.0.1 | 5923
Bloom filter allows skipping sstable 33255 | 20:16:26,699 | 127.0.0.1 | 5932
Bloom filter allows skipping sstable 33252 | 20:16:26,699 | 127.0.0.1 | 5938
Key cache hit for sstable 33247 | 20:16:26,699 | 127.0.0.1 | 5948
Seeking to partition beginning in data file | 20:16:26,699 | 127.0.0.1 | 5951
Bloom filter allows skipping sstable 33246 | 20:16:26,699 | 127.0.0.1 | 6072
Bloom filter allows skipping sstable 33243 | 20:16:26,699 | 127.0.0.1 | 6081
Key cache hit for sstable 33242 | 20:16:26,699 | 127.0.0.1 | 6092
Seeking to partition beginning in data file | 20:16:26,699 | 127.0.0.1 | 6095
Bloom filter allows skipping sstable 33240 | 20:16:26,699 | 127.0.0.1 | 6187
Key cache hit for sstable 33237 | 20:16:26,699 | 127.0.0.1 | 6198
Seeking to partition beginning in data file | 20:16:26,699 | 127.0.0.1 | 6201
Key cache hit for sstable 33235 | 20:16:26,699 | 127.0.0.1 | 6297
Seeking to partition beginning in data file | 20:16:26,699 | 127.0.0.1 | 6301
Bloom filter allows skipping sstable 33234 | 20:16:26,699 | 127.0.0.1 | 6393
Key cache hit for sstable 33229 | 20:16:26,699 | 127.0.0.1 | 6404
Seeking to partition beginning in data file | 20:16:26,699 | 127.0.0.1 | 6408
Bloom filter allows skipping sstable 33228 | 20:16:26,699 | 127.0.0.1 | 6496
Key cache hit for sstable 33227 | 20:16:26,699 | 127.0.0.1 | 6508
Seeking to partition beginning in data file | 20:16:26,699 | 127.0.0.1 | 6511
Key cache hit for sstable 33226 | 20:16:26,699 | 127.0.0.1 | 6601
Seeking to partition beginning in data file | 20:16:26,699 | 127.0.0.1 | 6605
Key cache hit for sstable 33225 | 20:16:26,700 | 127.0.0.1 | 6692
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 6696
Key cache hit for sstable 33223 | 20:16:26,700 | 127.0.0.1 | 6785
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 6789
Key cache hit for sstable 33221 | 20:16:26,700 | 127.0.0.1 | 6876
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 6880
Bloom filter allows skipping sstable 33219 | 20:16:26,700 | 127.0.0.1 | 6967
Key cache hit for sstable 33377 | 20:16:26,700 | 127.0.0.1 | 6978
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 6981
Key cache hit for sstable 33208 | 20:16:26,700 | 127.0.0.1 | 7071
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 7075
Key cache hit for sstable 33205 | 20:16:26,700 | 127.0.0.1 | 7161
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 7166
Bloom filter allows skipping sstable 33201 | 20:16:26,700 | 127.0.0.1 | 7251
Bloom filter allows skipping sstable 33200 | 20:16:26,700 | 127.0.0.1 | 7260
Key cache hit for sstable 33195 | 20:16:26,700 | 127.0.0.1 | 7276
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 7279
Bloom filter allows skipping sstable 33191 | 20:16:26,700 | 127.0.0.1 | 7363
Key cache hit for sstable 33190 | 20:16:26,700 | 127.0.0.1 | 7374
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 7377
Bloom filter allows skipping sstable 33189 | 20:16:26,700 | 127.0.0.1 | 7463
Key cache hit for sstable 33186 | 20:16:26,700 | 127.0.0.1 | 7474
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 7477
Key cache hit for sstable 33183 | 20:16:26,700 | 127.0.0.1 | 7563
Seeking to partition beginning in data file | 20:16:26,700 | 127.0.0.1 | 7567
Bloom filter allows skipping sstable 33182 | 20:16:26,701 | 127.0.0.1 | 7663
Bloom filter allows skipping sstable 33180 | 20:16:26,701 | 127.0.0.1 | 7672
Bloom filter allows skipping sstable 33178 | 20:16:26,701 | 127.0.0.1 | 7679
Bloom filter allows skipping sstable 33177 | 20:16:26,701 | 127.0.0.1 | 7686
也许最重要的是跟踪结束:
Merging data from memtables and 277 sstables | 20:21:29,186 | 127.0.0.1 | 607001
Read 3 live and 0 tombstoned cells | 20:21:29,186 | 127.0.0.1 | 607205
Request complete | 20:21:29,186 | 127.0.0.1 | 607714
答案 0 :(得分:2)
请查看跟踪以确认,但如果sdb,sdc和sdd正在旋转磁盘,您将看到正确的tps数量级,并且很可能是读取端绑定的随机磁盘I / O. / p>
如果是这种情况,那么您只有两个选项(任何系统,不是特定于Cassandra):
Cassandra可以为每个CPU内核执行大约3k-5K的操作(读取或写入),但前提是磁盘子系统不是限制因素。