说,我有一张桌子,有4列。我在里面写了一些数据。如果我尝试读取数据,则过程类似于this。我想了解一个特定的场景,其中,所有列(我尝试阅读的行)都存在于memtable中。 SSTables会检查这样一行的数据吗?我认为,在这种情况下,没有必要检查SSTables,因为memtable中的数据显然是最新的副本。因此,在这种情况下读取的内容应该比memtable不具有行或仅包含部分数据的内容更快。
我创建了一个表(user_data),并输入了一些导致创建2个SSTable的数据。在此之后,我插入了一个新行。我检查了数据目录并确保SSTable计数仍为2.这意味着我输入的新数据位于Memtable中。我设置了'跟踪'在cqlsh中然后选择相同的行。以下是输出:
Tracing session: de2e8ce0-cf1e-11e6-9318-a131a78ce29a
activity | timestamp | source | source_elapsed | client
----------------------------------------------------------------------------------------------+----------------------------+---------------+----------------+---------------
Execute CQL3 query | 2016-12-31 11:33:36.494000 | 172.16.129.67 | 0 | 172.16.129.67
Parsing select address,age from user_data where name='Kishan'; [Native-Transport-Requests-1] | 2016-12-31 11:33:36.495000 | 172.16.129.67 | 182 | 172.16.129.67
Preparing statement [Native-Transport-Requests-1] | 2016-12-31 11:33:36.495000 | 172.16.129.67 | 340 | 172.16.129.67
Executing single-partition query on user_data [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 | 693 | 172.16.129.67
Acquiring sstable references [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 | 765 | 172.16.129.67
Merging memtable contents [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 | 821 | 172.16.129.67
Read 1 live rows and 0 tombstone cells [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 | 1028 | 172.16.129.67
Request complete | 2016-12-31 11:33:36.495225 | 172.16.129.67 | 1225 | 172.16.129.67
我不明白"获取sstable引用的含义"这里。由于完整的数据位于Memtable中,因此,据我所知,没有必要检查SSTable。那么,这些参考究竟是什么?
答案 0 :(得分:3)
所有列(我试图读取的行)都存在于memtable.Will SSTables中,是否要检查这样一行的数据?
在这种特殊情况下, 它还会检查记忆中的可靠数据。
对于该列,它只会转到sstable(实际上先在行缓存中,然后是布隆过滤器,然后是 sstable )记忆中没有。
修改强>
要了解有关读取过程如何工作的更多信息,请深入了解cassandra源代码。让我们从跟踪日志开始,我们将逐行完成这些步骤:
让我们从这里开始:
Executing single-partition query on user_data [ReadStage-2]
您的选择查询是单个分区行查询,这是显而易见的。 Cassandra只需要从单个分区读取数据。让我们在这里跳转到相应的方法和java-doc,自我解释:
/**
* Queries both memtable and sstables to fetch the result of this query.
* <p>
* Please note that this method:
* 1) does not check the row cache.
* 2) does not apply the query limit, nor the row filter (and so ignore 2ndary indexes).
* Those are applied in {@link ReadCommand#executeLocally}.
* 3) does not record some of the read metrics (latency, scanned cells histograms) nor
* throws TombstoneOverwhelmingException.
* It is publicly exposed because there is a few places where that is exactly what we want,
* but it should be used only where you know you don't need thoses things.
* <p>
* Also note that one must have created a {@code ReadExecutionController} on the queried table and we require it as
* a parameter to enforce that fact, even though it's not explicitlly used by the method.
*/
public UnfilteredRowIterator queryMemtableAndDisk(ColumnFamilyStore cfs, ReadExecutionController executionController)
{
assert executionController != null && executionController.validForReadOn(cfs);
Tracing.trace("Executing single-partition query on {}", cfs.name);
return queryMemtableAndDiskInternal(cfs);
}
从avobe步骤我们发现,对于您的查询,它会调用queryMemtableAndDiskInternal(cfs);
此方法:
private UnfilteredRowIterator queryMemtableAndDiskInternal(ColumnFamilyStore cfs)
{
/*
* We have 2 main strategies:
* 1) We query memtables and sstables simulateneously. This is our most generic strategy and the one we use
* unless we have a names filter that we know we can optimize futher.
* 2) If we have a name filter (so we query specific rows), we can make a bet: that all column for all queried row
* will have data in the most recent sstable(s), thus saving us from reading older ones. This does imply we
* have a way to guarantee we have all the data for what is queried, which is only possible for name queries
* and if we have neither non-frozen collections/UDTs nor counters (indeed, for a non-frozen collection or UDT,
* we can't guarantee an older sstable won't have some elements that weren't in the most recent sstables,
* and counters are intrinsically a collection of shards and so have the same problem).
*/
if (clusteringIndexFilter() instanceof ClusteringIndexNamesFilter && !queriesMulticellType())
return queryMemtableAndSSTablesInTimestampOrder(cfs, (ClusteringIndexNamesFilter)clusteringIndexFilter());
...
...
我们在此评论中找到了答案:
<强> We have 2 main strategies:
1) We query memtables and sstables simulateneously. This is our most generic strategy and the one we use........
强>
Cassandra同时在memtable和sstables上查询。
之后如果我们跳到我们发现的queryMemtableAndSSTablesInTimestampOrder
方法:
/**
* Do a read by querying the memtable(s) first, and then each relevant sstables sequentially by order of the sstable
* max timestamp.
*
* This is used for names query in the hope of only having to query the 1 or 2 most recent query and then knowing nothing
* more recent could be in the older sstables (which we can only guarantee if we know exactly which row we queries, and if
* no collection or counters are included).
* This method assumes the filter is a {@code ClusteringIndexNamesFilter}.
*/
private UnfilteredRowIterator queryMemtableAndSSTablesInTimestampOrder(ColumnFamilyStore cfs, ClusteringIndexNamesFilter filter)
{
Tracing.trace("Acquiring sstable references");
ColumnFamilyStore.ViewFragment view = cfs.select(View.select(SSTableSet.LIVE, partitionKey()));
ImmutableBTreePartition result = null;
Tracing.trace("Merging memtable contents");
.... // then it also looks into sstable on timestamp order.
从上面的部分我们已经找到了我们的最后两个跟踪日志:
Acquiring sstable references [ReadStage-2]
Merging memtable contents [ReadStage-2]
希望这有帮助。