Question

我在变量中有SELECT查询的结果，现在我想逐行遍历查询结果来进行一些处理，比如找一个特定的模式。例如，模式可能如下：

a, b, c, d, e
b, c, d, e, f
c, d, e, f, g

CSV中SELECT查询的结果可能是：

1, 2, 3, 4, 5
3, 4, 5, 6, 7
a, b, c, d, e
b, c, d, e, f
c, d, e, f, g
5, 6, 7, 8, 9

我已经看过一些使用自定义提取器的PROCESS语句，但是这样做的方法是什么？我不确定这个过程和提取器是如何工作的。

https://msdn.microsoft.com/en-us/library/azure/mt621322.aspx

感谢您的帮助。

Answer 1

我认为你不需要迭代。更多基于集合的方法是否适合您？试试我创建的这个示例U-SQL脚本。基本上如果匹配，结果将在文件中，如果没有匹配，则文件将为空。

// Set the search pattern
@pattern = 
    SELECT *
    FROM ( VALUES 
            ( "a", "b", "c", "d", "e" ),
            ( "b", "c", "d", "e", "f" ),
            ( "c", "d", "e", "f", "g" )
           ) AS t (col1, col2, col3, col4, col5 );


// Get the file to search
@input =
    EXTRACT col1 string,
            col2 string,
            col3 string,
            col4 string,
            col5 string
    FROM "/input/input.csv"
    USING Extractors.Csv();


// Add rowIds
@pattern =
    SELECT ROW_NUMBER() OVER() AS rowId, *
    FROM @pattern;

@input =
    SELECT ROW_NUMBER() OVER() AS rowId, *
    FROM @input;


// Check the same rows appear in the same order
@temp =
    SELECT i.rowId,
           p.rowId == null ? 0 : ROW_NUMBER() OVER( ORDER BY p.rowId ) AS rowId2    // Restarts the row numbering when there is a match
    FROM @input AS i
         LEFT OUTER JOIN
             @pattern AS p
         ON i.col1 == p.col1
            AND i.col2 == p.col2
            AND i.col3 == p.col3
            AND i.col4 == p.col4
            AND i.col5 == p.col5;


@output =
    SELECT p.*
    FROM @pattern AS p
         INNER JOIN
             @temp AS t
         ON p.rowId == t.rowId2;


@pattenRecords =
    SELECT COUNT( * ) AS records
    FROM @pattern;


@records =
    SELECT COUNT( * ) AS records
    FROM @output;


// Join criteria mean output file will be empty if there has not been a match
@output =
    SELECT o.*
    FROM @output AS o
       CROSS JOIN @records AS r
       INNER JOIN
          (
          SELECT *
          FROM @pattenRecords
          INTERSECT
          SELECT *
          FROM @records
          ) AS t ON r.records == t.records;



// Output results
OUTPUT @output
    TO "/output/output.csv"
USING Outputters.Csv();

也许有一种更简单的方法。

如何使用U-SQL

1 个答案: