Linq-to-SQL：提高性能（RANK OVER PARTITION）

Question

简

我正在努力将SQL脚本转换为Linq。此脚本使用RANK() OVER (PARTITION BY...

然而，当Linq将过程传递给SQL时，我留下了一个奇怪的结果，这导致对表的读取次数增加，并且与 simpler相比增加了查询时间 Linq版本或原始SQL。

我意识到持续时间结果略微不同，但是，想法是尽可能使用 Linq to SQL 。我正在尝试优化生成的 Linq to SQL 代码以提高性能。

平均持续时间基于以下每个脚本的1000次迭代。它们已四舍五入到最接近50的倍数。

注意：平均持续时间以微秒为单位。

研究

我在this帖子上看到了答案。我不相信我的查询特别昂贵，因为它相当简单和直接。然而，它创建了一个复杂的查询，显示性能更差。

代码

SQL

SQL脚本可以用几种不同的方式编写，因此我将提供以下两种方法来成功完成以下编写此脚本。两种方法主要是相同的。一个使用CTE，而另一个使用嵌套查询。

方法1（CTE）

阅读：2
平均持续时间：650

DECLARE @date DATETIME = '2017-09-20'

;WITH Currency AS (
    SELECT 
            CurrencyType,
            AsOfDate,
            ConvFactor,
            RANK() OVER (PARTITION BY CurrencyType ORDER BY AsOfDate DESC) AS ConversionRank
        FROM CurrencyDtl
        WHERE AsOfDate <= @date
)

SELECT
        CurrencyType,
        AsOfDate,
        ConvFactor
    FROM Currency
    WHERE ConversionRank = 1

方法2（嵌套查询）

阅读：2
平均持续时间：600

DECLARE @date DATETIME = '2017-09-20'

SELECT
        cd.CurrencyType,
        cd.AsOfDate,
        cd.ConvFactor
    FROM (
        SELECT 
                cd.CurrencyType,
                cd.AsOfDate,
                cd.ConvFactor,
                RANK() OVER (PARTITION BY cd.CurrencyType ORDER BY cd.AsOfDate DESC) AS ConversionRank
            FROM CurrencyDtl AS cd
            WHERE cd.AsOfDate <= @date
    ) AS cd
    WHERE cd.ConversionRank = 1

LINQ的

方法1

问题：实际上并没有返回我需要的值：第二个Linq方法确实返回了我需要的值但成本更高。

阅读：6
平均持续时间：900

LINQ的

MyDataContext db = new MyDataContext();
Table<CurrencyDtl> CurrencyDtl = db.GetTable<CurrencyDtl>();

DateTime date = DateTime.ParseExact("2017-09-20", "yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture);

var q = (from cd in CurrencyDtl
         where cd.AsOfDate <= date
         group cd by cd.CurrencyType into grp
         select grp.OrderByDescending(g => g.AsOfDate).First());

SQL （Linq生成）

exec sp_executesql N'SELECT [t3].[test], [t3].[CurrencyType], [t3].[AsOfDate], [t3].[ConvFactor], [t3].[CurrencyDtlKey]
FROM (
    SELECT [t0].[CurrencyType]
    FROM [dbo].[CurrencyDtl] AS [t0]
    WHERE [t0].[AsOfDate] <= @p0
    GROUP BY [t0].[CurrencyType]
    ) AS [t1]
OUTER APPLY (
    SELECT TOP (1) 1 AS [test], [t2].[CurrencyType], [t2].[AsOfDate], [t2].[ConvFactor], [t2].[CurrencyDtlKey]
    FROM [dbo].[CurrencyDtl] AS [t2]
    WHERE ((([t1].[CurrencyType] IS NULL) AND ([t2].[CurrencyType] IS NULL)) OR (([t1].[CurrencyType] IS NOT NULL) AND ([t2].[CurrencyType] IS NOT NULL) AND ([t1].[CurrencyType] = [t2].[CurrencyType]))) AND ([t2].[AsOfDate] <= @p0)
    ORDER BY [t2].[AsOfDate] DESC
    ) AS [t3]
ORDER BY [t3].[AsOfDate] DESC',N'@p0 datetime',@p0='2017-09-20 00:00:00'

方法2

阅读：14
平均持续时间：1200

LINQ的

MyDataContext db = new MyDataContext();
Table<CurrencyDtl> CurrencyDtl = db.GetTable<CurrencyDtl>();

DateTime date = DateTime.ParseExact("2017-09-20", "yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture);

var q = (from cd in CurrencyDtl
         where cd.AsOfDate <= date
         group cd by cd.CurrencyType into grp
         let c = grp.OrderByDescending(g => g.AsOfDate).First()
         select new
         {
             c.CurrencyType,
             c.AsOfDate,
             c.ConvFactor
         });

SQL （Linq生成）

exec sp_executesql N'SELECT (
    SELECT [t3].[CurrencyType]
    FROM (
        SELECT TOP (1) [t2].[CurrencyType]
        FROM [dbo].[CurrencyDtl] AS [t2]
        WHERE ((([t1].[CurrencyType] IS NULL) AND ([t2].[CurrencyType] IS NULL)) OR (([t1].[CurrencyType] IS NOT NULL) AND ([t2].[CurrencyType] IS NOT NULL) AND ([t1].[CurrencyType] = [t2].[CurrencyType]))) AND ([t2].[AsOfDate] <= @p0)
        ORDER BY [t2].[AsOfDate] DESC
        ) AS [t3]
    ) AS [CurrencyType], (
    SELECT [t5].[AsOfDate]
    FROM (
        SELECT TOP (1) [t4].[AsOfDate]
        FROM [dbo].[CurrencyDtl] AS [t4]
        WHERE ((([t1].[CurrencyType] IS NULL) AND ([t4].[CurrencyType] IS NULL)) OR (([t1].[CurrencyType] IS NOT NULL) AND ([t4].[CurrencyType] IS NOT NULL) AND ([t1].[CurrencyType] = [t4].[CurrencyType]))) AND ([t4].[AsOfDate] <= @p0)
        ORDER BY [t4].[AsOfDate] DESC
        ) AS [t5]
    ) AS [AsOfDate], (
    SELECT [t7].[ConvFactor]
    FROM (
        SELECT TOP (1) [t6].[ConvFactor]
        FROM [dbo].[CurrencyDtl] AS [t6]
        WHERE ((([t1].[CurrencyType] IS NULL) AND ([t6].[CurrencyType] IS NULL)) OR (([t1].[CurrencyType] IS NOT NULL) AND ([t6].[CurrencyType] IS NOT NULL) AND ([t1].[CurrencyType] = [t6].[CurrencyType]))) AND ([t6].[AsOfDate] <= @p0)
        ORDER BY [t6].[AsOfDate] DESC
        ) AS [t7]
    ) AS [ConvFactor]
FROM (
    SELECT [t0].[CurrencyType]
    FROM [dbo].[CurrencyDtl] AS [t0]
    WHERE [t0].[AsOfDate] <= @p0
    GROUP BY [t0].[CurrencyType]
    ) AS [t1]',N'@p0 datetime',@p0='2017-09-20 00:00:00'

问题

1. 如何减少Linq-gerenated查询的持续时间？
2. 如何减少Linq生成的查询执行的读取次数？
3. 为什么我的方法2 Linq生成的查询如此错综复杂？
4. 您是否有任何提示可以提高Linq生成的更多查询的效果？

P.S。感谢您抽出宝贵时间阅读这篇冗长而详细的文章。

Answer 1

我认为你正在对抗随机数字。我在20M行表上测试了它。 linq查询耗时2秒，但等级约为20秒。

如果表现如此重要，我会推荐不同的东西。只需将每天所有货币的汇率存入表格。然后你只需要查询一天的行，这将尽可能高效。它还可以保证您在当天使用有效值而不是过时的值。

Answer 2

简

继续回答我自己的问题。您可以在分组中选择所需的行，而不是在选择中执行此操作。使用RANK OVER PARTITION BY似乎会创建重复的子查询（至少对于MyDataContext db = new MyDataContext(); Table<CurrencyDtl> CurrencyDtl = db.GetTable<CurrencyDtl>(); DateTime date = DateTime.ParseExact("2017-09-20", "yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture); var q = (from cd in CurrencyDtl where cd.AsOfDate <= date group new { cd.CurrencyType, cd.AsOfDate, cd.ConvFactor } by cd.CurrencyType into grp select grp.OrderByDescending(g => g.AsOfDate).First()); 模拟）。

此方法可缩短持续时间，但不会减少读数。如果有人有更好的解决方案，我会非常乐意接受它，但就目前而言，这似乎是最好的方法。

代码

阅读：6
平均持续时间：700

LINQ的

exec sp_executesql N'SELECT [t3].[test], [t3].[CurrencyType], [t3].[AsOfDate], [t3].[ConvFactor]
FROM (
    SELECT [t0].[CurrencyType]
    FROM [dbo].[CurrencyDtl] AS [t0]
    WHERE [t0].[AsOfDate] <= @p0
    GROUP BY [t0].[CurrencyType]
    ) AS [t1]
OUTER APPLY (
    SELECT TOP (1) 1 AS [test], [t2].[CurrencyType], [t2].[AsOfDate], [t2].[ConvFactor]
    FROM [dbo].[CurrencyDtl] AS [t2]
    WHERE ((([t1].[CurrencyType] IS NULL) AND ([t2].[CurrencyType] IS NULL)) OR (([t1].[CurrencyType] IS NOT NULL) AND ([t2].[CurrencyType] IS NOT NULL) AND ([t1].[CurrencyType] = [t2].[CurrencyType]))) AND ([t2].[AsOfDate] <= @p0)
    ORDER BY [t2].[AsOfDate] DESC
    ) AS [t3]
ORDER BY [t3].[AsOfDate] DESC',N'@p0 datetime',@p0='2017-09-20 00:00:00'

SQL

for ((i=0;i<1000;i++)); do <some command> <formatted string with i>; done.