Question

我对我们的数据仓库有一个运行时间非常长的查询，需要了解如何优化它。 [AssessorLogDW]。[dbo]。[Log]表具有1.05亿行。这是查询。

select a.Nationid,count(a.Nationid)as nationcount from 
    (select replace(replace(convert(nvarchar(MAX),[parameters]), '<root><nationid>', ''),'</nationid></root>','') as Nationid
    from [AssessorLogDW].[dbo].[Log]
    where username not like '%erieri.com' and "Parameters" is not null) as a
group by a.Nationid

这是表上的索引。

位于主时间戳上的非集群式IX_Log
IXM_Log_1非群集位于PRIMARY类型上
IXIM_Log_3非群集位于PRIMARY AppCode
非群集的IX_Log_4位于PRIMARY ServiceNamespace
非群集的IX_Log_5位于PRIMARY ServiceName
非群集的IX_Log_6位于主要方法名上
非群集IX_Log_7位于主要持续时间
非聚集的IX_Log_MethodName位于PRIMARY SubscriptionCode上，MethodName
非聚集的IX_Log_UserName位于PRIMARY UserName，SubscriptionCode，MethodName，Timestamp
非聚集的IX_Log_UserNameDate位于PRIMARY用户名，时间戳记，SubscriptionCode上
ix_TypeAppCodeMethodName非聚集在PRIMARY Type，AppCode，MethodName，Timestamp上
位于PRIMARY DWid上的PK_Log集群的唯一主键

这是执行计划：

<ShowPlanXML xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan" Version="1.3.1" Build="12.0.5207.0"><BatchSequence><Batch><Statements><StmtSimple StatementText="select a.Nationid,count(a.Nationid)as nationcount from &#xD;&#xA;    (select replace(replace(convert(nvarchar(MAX),[parameters]), '&lt;root&gt;&lt;nationid&gt;', ''),'&lt;/nationid&gt;&lt;/root&gt;','') as Nationid&#xD;&#xA;    from [AssessorLogDW].[dbo].[Log]&#xD;&#xA;    where username not like '%erieri.com' and &quot;Parameters&quot; is not null) as a&#xD;&#xA;group by a.Nationid" StatementId="1" StatementCompId="1" StatementType="SELECT" RetrievedFromCache="true" StatementSubTreeCost="18373.7" StatementEstRows="8240.4" StatementOptmLevel="FULL" QueryHash="0x288FAF0B2CDAA4D5" QueryPlanHash="0xBC649190D004A5F1" CardinalityEstimationModelVersion="70"><StatementSetOptions QUOTED_IDENTIFIER="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" ANSI_NULLS="true" ANSI_PADDING="true" ANSI_WARNINGS="true" NUMERIC_ROUNDABORT="false" /><QueryPlan CachedPlanSize="56" CompileTime="27" CompileCPU="12" CompileMemory="672"><ThreadStat Branches="2" /><MemoryGrantInfo SerialRequiredMemory="1536" SerialDesiredMemory="83680" /><OptimizerHardwareDependentProperties EstimatedAvailableMemoryGrant="256000" EstimatedPagesCached="64000" EstimatedAvailableDegreeOfParallelism="2" MaxCompileMemory="2424384" /><TraceFlags IsCompileTime="1"><TraceFlag Value="845" Scope="Global" /><TraceFlag Value="1117" Scope="Global" /><TraceFlag Value="1118" Scope="Global" /><TraceFlag Value="3226" Scope="Global" /><TraceFlag Value="4199" Scope="Global" /></TraceFlags><RelOp NodeId="1" PhysicalOp="Compute Scalar" LogicalOp="Compute Scalar" EstimateRows="16480.8" EstimateIO="0" EstimateCPU="0.00164808" AvgRowSize="4039" EstimatedTotalSubtreeCost="18373.7" Parallel="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Column="Expr1002" /><ColumnReference Column="Expr1003" /></OutputList><ComputeScalar><DefinedValues><DefinedValue><ColumnReference Column="Expr1003" /><ScalarOperator ScalarString="CONVERT_IMPLICIT(int,[globalagg1009],0)"><Convert DataType="int" Style="0" Implicit="1"><ScalarOperator><Identifier><ColumnReference Column="globalagg1009" /></Identifier></ScalarOperator></Convert></ScalarOperator></DefinedValue></DefinedValues><RelOp NodeId="2" PhysicalOp="Parallelism" LogicalOp="Gather Streams" EstimateRows="16480.8" EstimateIO="0" EstimateCPU="2.34046" AvgRowSize="4043" EstimatedTotalSubtreeCost="18373.7" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Column="Expr1002" /><ColumnReference Column="globalagg1009" /></OutputList><Parallelism><RelOp NodeId="3" PhysicalOp="Stream Aggregate" LogicalOp="Aggregate" EstimateRows="16480.8" EstimateIO="0" EstimateCPU="0.00906444" AvgRowSize="4043" EstimatedTotalSubtreeCost="18371.3" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Column="Expr1002" /><ColumnReference Column="globalagg1009" /></OutputList><StreamAggregate><DefinedValues><DefinedValue><ColumnReference Column="globalagg1009" /><ScalarOperator ScalarString="SUM([partialagg1008])"><Aggregate Distinct="0" AggType="SUM"><ScalarOperator><Identifier><ColumnReference Column="partialagg1008" /></Identifier></ScalarOperator></Aggregate></ScalarOperator></DefinedValue></DefinedValues><GroupBy><ColumnReference Column="Expr1002" /></GroupBy><RelOp NodeId="4" PhysicalOp="Sort" LogicalOp="Sort" EstimateRows="16480.8" EstimateIO="0.00563063" EstimateCPU="0.529902" AvgRowSize="4043" EstimatedTotalSubtreeCost="18371.3" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Column="Expr1002" /><ColumnReference Column="partialagg1008" /></OutputList><MemoryFractions Input="1" Output="1" /><Sort Distinct="0"><OrderBy><OrderByColumn Ascending="1"><ColumnReference Column="Expr1002" /></OrderByColumn></OrderBy><RelOp NodeId="5" PhysicalOp="Parallelism" LogicalOp="Repartition Streams" EstimateRows="16480.8" EstimateIO="0" EstimateCPU="3.98045" AvgRowSize="4043" EstimatedTotalSubtreeCost="18370.8" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Column="Expr1002" /><ColumnReference Column="partialagg1008" /></OutputList><Parallelism PartitioningType="Hash"><PartitionColumns><ColumnReference Column="Expr1002" /></PartitionColumns><RelOp NodeId="6" PhysicalOp="Hash Match" LogicalOp="Partial Aggregate" EstimateRows="16480.8" EstimateIO="0" EstimateCPU="276.148" AvgRowSize="4043" EstimatedTotalSubtreeCost="18366.8" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Column="Expr1002" /><ColumnReference Column="partialagg1008" /></OutputList><MemoryFractions Input="0" Output="0" /><Hash><DefinedValues><DefinedValue><ColumnReference Column="partialagg1008" /><ScalarOperator ScalarString="COUNT(replace(replace([Expr1006],N'&lt;root&gt;&lt;nationid&gt;',CONVERT_IMPLICIT(nvarchar(max),'',0)),N'&lt;/nationid&gt;&lt;/root&gt;',CONVERT_IMPLICIT(nvarchar(max),'',0)))"><Aggregate Distinct="0" AggType="COUNT_BIG"><ScalarOperator><Intrinsic FunctionName="replace"><ScalarOperator><Intrinsic FunctionName="replace"><ScalarOperator><Identifier><ColumnReference Column="Expr1006" /></Identifier></ScalarOperator><ScalarOperator><Const ConstValue="N'&lt;root&gt;&lt;nationid&gt;'" /></ScalarOperator><ScalarOperator><Identifier><ColumnReference Column="ConstExpr1005"><ScalarOperator><Convert DataType="nvarchar(max)" Length="2147483647" Style="0" Implicit="1"><ScalarOperator><Const ConstValue="''" /></ScalarOperator></Convert></ScalarOperator></ColumnReference></Identifier></ScalarOperator></Intrinsic></ScalarOperator><ScalarOperator><Const ConstValue="N'&lt;/nationid&gt;&lt;/root&gt;'" /></ScalarOperator><ScalarOperator><Identifier><ColumnReference Column="ConstExpr1004"><ScalarOperator><Convert DataType="nvarchar(max)" Length="2147483647" Style="0" Implicit="1"><ScalarOperator><Const ConstValue="''" /></ScalarOperator></Convert></ScalarOperator></ColumnReference></Identifier></ScalarOperator></Intrinsic></ScalarOperator></Aggregate></ScalarOperator></DefinedValue></DefinedValues><HashKeysBuild><ColumnReference Column="Expr1002" /></HashKeysBuild><BuildResidual><ScalarOperator ScalarString="[Expr1002] = [Expr1002]"><Compare CompareOp="IS"><ScalarOperator><Identifier><ColumnReference Column="Expr1002" /></Identifier></ScalarOperator><ScalarOperator><Identifier><ColumnReference Column="Expr1002" /></Identifier></ScalarOperator></Compare></ScalarOperator></BuildResidual><RelOp NodeId="7" PhysicalOp="Compute Scalar" LogicalOp="Compute Scalar" EstimateRows="6.79042e+007" EstimateIO="0" EstimateCPU="3.39521" AvgRowSize="8061" EstimatedTotalSubtreeCost="18090.7" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Column="Expr1002" /><ColumnReference Column="Expr1006" /></OutputList><ComputeScalar><DefinedValues><DefinedValue><ColumnReference Column="Expr1002" /><ScalarOperator ScalarString="replace(replace([Expr1007],N'&lt;root&gt;&lt;nationid&gt;',CONVERT_IMPLICIT(nvarchar(max),'',0)),N'&lt;/nationid&gt;&lt;/root&gt;',CONVERT_IMPLICIT(nvarchar(max),'',0))"><Intrinsic FunctionName="replace"><ScalarOperator><Intrinsic FunctionName="replace"><ScalarOperator><Identifier><ColumnReference Column="Expr1007" /></Identifier></ScalarOperator><ScalarOperator><Const ConstValue="N'&lt;root&gt;&lt;nationid&gt;'" /></ScalarOperator><ScalarOperator><Identifier><ColumnReference Column="ConstExpr1005"><ScalarOperator><Convert DataType="nvarchar(max)" Length="2147483647" Style="0" Implicit="1"><ScalarOperator><Const ConstValue="''" /></ScalarOperator></Convert></ScalarOperator></ColumnReference></Identifier></ScalarOperator></Intrinsic></ScalarOperator><ScalarOperator><Const ConstValue="N'&lt;/nationid&gt;&lt;/root&gt;'" /></ScalarOperator><ScalarOperator><Identifier><ColumnReference Column="ConstExpr1004"><ScalarOperator><Convert DataType="nvarchar(max)" Length="2147483647" Style="0" Implicit="1"><ScalarOperator><Const ConstValue="''" /></ScalarOperator></Convert></ScalarOperator></ColumnReference></Identifier></ScalarOperator></Intrinsic></ScalarOperator></DefinedValue></DefinedValues><RelOp NodeId="8" PhysicalOp="Compute Scalar" LogicalOp="Compute Scalar" EstimateRows="6.79042e+007" EstimateIO="0" EstimateCPU="3.39521" AvgRowSize="8061" EstimatedTotalSubtreeCost="18087.3" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Column="Expr1006" /><ColumnReference Column="Expr1007" /></OutputList><ComputeScalar><DefinedValues><DefinedValue><ColumnReference Column="Expr1006" /><ScalarOperator ScalarString="CONVERT(nvarchar(max),[AssessorLogDW].[dbo].[Log].[Parameters],0)"><Convert DataType="nvarchar(max)" Length="2147483647" Style="0" Implicit="0"><ScalarOperator><Identifier><ColumnReference Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Column="Parameters" /></Identifier></ScalarOperator></Convert></ScalarOperator></DefinedValue><DefinedValue><ColumnReference Column="Expr1007" /><ScalarOperator ScalarString="CONVERT(nvarchar(max),[AssessorLogDW].[dbo].[Log].[Parameters],0)"><Convert DataType="nvarchar(max)" Length="2147483647" Style="0" Implicit="0"><ScalarOperator><Identifier><ColumnReference Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Column="Parameters" /></Identifier></ScalarOperator></Convert></ScalarOperator></DefinedValue></DefinedValues><RelOp NodeId="9" PhysicalOp="Filter" LogicalOp="Filter" EstimateRows="6.79042e+007" EstimateIO="0" EstimateCPU="72.688" AvgRowSize="4035" EstimatedTotalSubtreeCost="18083.9" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Column="Parameters" /></OutputList><Filter StartupExpression="0"><RelOp NodeId="10" PhysicalOp="Index Scan" LogicalOp="Index Scan" EstimateRows="6.79042e+007" EstimateIO="17953.2" EstimateCPU="57.9398" AvgRowSize="4060" EstimatedTotalSubtreeCost="18011.2" TableCardinality="1.05345e+008" Parallel="1" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row"><OutputList><ColumnReference Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Column="Parameters" /></OutputList><Warnings><ColumnsWithNoStatistics><ColumnReference Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Column="Parameters" /></ColumnsWithNoStatistics></Warnings><IndexScan Ordered="0" ForcedIndex="0" ForceSeek="0" ForceScan="0" NoExpandHint="0" Storage="RowStore"><DefinedValues><DefinedValue><ColumnReference Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Column="Parameters" /></DefinedValue></DefinedValues><Object Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Index="[ix_TypeAppCodeMethodName]" IndexKind="NonClustered" Storage="RowStore" /><Predicate><ScalarOperator ScalarString="NOT [AssessorLogDW].[dbo].[Log].[UserName] like '%erieri.com'"><Logical Operation="NOT"><ScalarOperator><Intrinsic FunctionName="like"><ScalarOperator><Identifier><ColumnReference Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Column="UserName" /></Identifier></ScalarOperator><ScalarOperator><Const ConstValue="'%erieri.com'" /></ScalarOperator></Intrinsic></ScalarOperator></Logical></ScalarOperator></Predicate></IndexScan></RelOp><Predicate><ScalarOperator ScalarString="[AssessorLogDW].[dbo].[Log].[Parameters] IS NOT NULL"><Compare CompareOp="IS NOT"><ScalarOperator><Identifier><ColumnReference Database="[AssessorLogDW]" Schema="[dbo]" Table="[Log]" Column="Parameters" /></Identifier></ScalarOperator><ScalarOperator><Const ConstValue="NULL" /></ScalarOperator></Compare></ScalarOperator></Predicate></Filter></RelOp></ComputeScalar></RelOp></ComputeScalar></RelOp></Hash></RelOp></Parallelism></RelOp></Sort></RelOp></StreamAggregate></RelOp></Parallelism></RelOp></ComputeScalar></RelOp></QueryPlan></StmtSimple></Statements></Batch></BatchSequence></ShowPlanXML>

Answer 1

您可以使用CROSS APPLY来避免subquery：

select Nationid, count(*) as nationcount 
from [AssessorLogDW].[dbo].[Log] l1 cross apply
     ( values (replace(replace(convert(nvarchar(max), [parameters]), '<root><nationid>', ''),'</nationid></root>','') 
              )
     ) l2(Nationid)
where username not like '%erieri.com' and 
      [parameters] is not null
group by Nationid;

Answer 2

@rtoyo是正确的。问题是通配符。该专栏的实际可能性是什么？也许不用前导通配符也可以得到相同的结果。如果erieri.com是域，则可以获取子域列表吗？如果是这样，您可以使用它在名称的该部分上构造一个联接。

Answer 3

您没有说此查询的选择性是什么，因此我假设它是一个允许使用索引的好方法。我会做两件事：

您发布的计划警告parameters表的Log列没有统计信息。优化时，您需要确保统计信息是最新的。因此，首先运行：
```
update statistics [AssessorLogDW].[dbo].[Log];
```

然后，为了从根本上提高查询速度，您需要最小化“索引范围扫描”，以便遍历尽可能少的索引叶节点。如果速度对您真的很重要，我会：

创建虚拟列：

alter table log add reverse_username as reverse(username);

创建两个使用虚拟列的新索引（让优化器选择最佳索引）：

create index ix1_log on log (parameters, reverse_username);
create index ix2_log on log (reverse_username, parameters);

稍微更改查询以充分利用新索引：

select a.Nationid,count(a.Nationid)as nationcount from 
    (select replace(replace(convert(nvarchar(MAX),[parameters]),
      '<root><nationid>', ''),'</nationid></root>','') as Nationid
    from [AssessorLogDW].[dbo].[Log]
    where reverse_username not like reverse('%erieri.com')
      and "Parameters" is not null) as a
group by a.Nationid

诀窍是使用反向用户名进行搜索，以便将通配符放在末尾。这就开辟了以较小的“范围”充分利用索引的可能性。索引遍历将很快而且很短。

Answer 4

您可以在SQL Server中创建索引视图。 https://www.codeproject.com/Articles/199058/SQL-Server-Indexed-Views-Speed-Up-Your-Select-Quer

寻求优化此查询

4 个答案: