卡桑德拉阅读过程

时间:2018-04-11 08:01:42

标签: cassandra cassandra-3.0

说,我有一张桌子,有4列。我在里面写了一些数据。如果我尝试读取数据,则过程类似于this。我想了解一个特定的场景,其中,所有列(我尝试阅读的行)都存在于memtable中。 SSTables会检查这样一行的数据吗?我认为,在这种情况下,没有必要检查SSTables,因为memtable中的数据显然是最新的副本。因此,在这种情况下读取的内容应该比memtable不具有行或仅包含部分数据的内容更快。

我创建了一个表(user_data),并输入了一些导致创建2个SSTable的数据。在此之后,我插入了一个新行。我检查了数据目录并确保SSTable计数仍为2.这意味着我输入的新数据位于Memtable中。我设置了'跟踪'在cqlsh中然后选择相同的行。以下是输出:

Tracing session: de2e8ce0-cf1e-11e6-9318-a131a78ce29a

 activity                                                                                     | timestamp                  | source        | source_elapsed | client
----------------------------------------------------------------------------------------------+----------------------------+---------------+----------------+---------------
                                                                           Execute CQL3 query | 2016-12-31 11:33:36.494000 | 172.16.129.67 |              0 | 172.16.129.67
 Parsing select address,age from user_data where name='Kishan'; [Native-Transport-Requests-1] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            182 | 172.16.129.67
                                            Preparing statement [Native-Transport-Requests-1] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            340 | 172.16.129.67
                                  Executing single-partition query on user_data [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            693 | 172.16.129.67
                                                   Acquiring sstable references [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            765 | 172.16.129.67
                                                      Merging memtable contents [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |            821 | 172.16.129.67
                                         Read 1 live rows and 0 tombstone cells [ReadStage-2] | 2016-12-31 11:33:36.495000 | 172.16.129.67 |           1028 | 172.16.129.67
                                                                             Request complete | 2016-12-31 11:33:36.495225 | 172.16.129.67 |           1225 | 172.16.129.67

我不明白"获取sstable引用的含义"这里。由于完整的数据位于Memtable中,因此,据我所知,没有必要检查SSTable。那么,这些参考究竟是什么?

1 个答案:

答案 0 :(得分:3)

  

所有列(我试图读取的行)都存在于memtable.Will SSTables中,是否要检查这样一行的数据?

在这种特殊情况下, 它还会检查记忆中的可靠数据。

对于该列,它只会转到sstable(实际上先在行缓存中,然后是布隆过滤器,然后是 sstable )记忆中没有。

修改

要了解有关读取过程如何工作的更多信息,请深入了解cassandra源代码。让我们从跟踪日志开始,我们将逐行完成这些步骤:

让我们从这里开始:

  

Executing single-partition query on user_data [ReadStage-2]

您的选择查询是单个分区行查询,这是显而易见的。 Cassandra只需要从单个分区读取数据。让我们在这里跳转到相应的方法和java-doc,自我解释:

/**
 * Queries both memtable and sstables to fetch the result of this query.
 * <p>
 * Please note that this method:
 *   1) does not check the row cache.
 *   2) does not apply the query limit, nor the row filter (and so ignore 2ndary indexes).
 *      Those are applied in {@link ReadCommand#executeLocally}.
 *   3) does not record some of the read metrics (latency, scanned cells histograms) nor
 *      throws TombstoneOverwhelmingException.
 * It is publicly exposed because there is a few places where that is exactly what we want,
 * but it should be used only where you know you don't need thoses things.
 * <p>
 * Also note that one must have created a {@code ReadExecutionController} on the queried table and we require it as
 * a parameter to enforce that fact, even though it's not explicitlly used by the method.
 */
public UnfilteredRowIterator queryMemtableAndDisk(ColumnFamilyStore cfs, ReadExecutionController executionController)
{
    assert executionController != null && executionController.validForReadOn(cfs);
    Tracing.trace("Executing single-partition query on {}", cfs.name);

    return queryMemtableAndDiskInternal(cfs);
}

从avobe步骤我们发现,对于您的查询,它会调用queryMemtableAndDiskInternal(cfs);此方法:

private UnfilteredRowIterator queryMemtableAndDiskInternal(ColumnFamilyStore cfs)
    {
        /*
         * We have 2 main strategies:
         *   1) We query memtables and sstables simulateneously. This is our most generic strategy and the one we use
         *      unless we have a names filter that we know we can optimize futher.
         *   2) If we have a name filter (so we query specific rows), we can make a bet: that all column for all queried row
         *      will have data in the most recent sstable(s), thus saving us from reading older ones. This does imply we
         *      have a way to guarantee we have all the data for what is queried, which is only possible for name queries
         *      and if we have neither non-frozen collections/UDTs nor counters (indeed, for a non-frozen collection or UDT,
         *      we can't guarantee an older sstable won't have some elements that weren't in the most recent sstables,
         *      and counters are intrinsically a collection of shards and so have the same problem).
         */
        if (clusteringIndexFilter() instanceof ClusteringIndexNamesFilter && !queriesMulticellType())
            return queryMemtableAndSSTablesInTimestampOrder(cfs, (ClusteringIndexNamesFilter)clusteringIndexFilter());
        ...
        ...

我们在此评论中找到了答案:

<强> We have 2 main strategies: 1) We query memtables and sstables simulateneously. This is our most generic strategy and the one we use........

Cassandra同时在memtable和sstables上查询。

之后如果我们跳到我们发现的queryMemtableAndSSTablesInTimestampOrder方法:

/**
 * Do a read by querying the memtable(s) first, and then each relevant sstables sequentially by order of the sstable
 * max timestamp.
 *
 * This is used for names query in the hope of only having to query the 1 or 2 most recent query and then knowing nothing
 * more recent could be in the older sstables (which we can only guarantee if we know exactly which row we queries, and if
 * no collection or counters are included).
 * This method assumes the filter is a {@code ClusteringIndexNamesFilter}.
 */
private UnfilteredRowIterator queryMemtableAndSSTablesInTimestampOrder(ColumnFamilyStore cfs, ClusteringIndexNamesFilter filter)
{
    Tracing.trace("Acquiring sstable references");
    ColumnFamilyStore.ViewFragment view = cfs.select(View.select(SSTableSet.LIVE, partitionKey()));

    ImmutableBTreePartition result = null;

    Tracing.trace("Merging memtable contents");
    .... // then it also looks into sstable on timestamp order.

从上面的部分我们已经找到了我们的最后两个跟踪日志:

  

Acquiring sstable references [ReadStage-2]

     

Merging memtable contents [ReadStage-2]

希望这有帮助。

相关链接: Source: SinglePartitionReadCommand.java