Question

有人能告诉我如何将hive查询转换为抽象语法树吗？例如：select * from cust where cust_num = 100; 我怎么能把它转换成AST？以及如何将此AST转换为QB树？请帮忙。提前谢谢。

Answer 1

您可以将EXPLAIN命令与EXTENDED一起使用。假设我有一个名为demo的表，其列为n1，然后发出Explain会给我这个：

hive> EXPLAIN EXTENDED select * from demo where n1='aaa';
OK
ABSTRACT SYNTAX TREE:
  (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME demo))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_WHERE (= (TOK_TABLE_OR_COL n1) 'aaa'))))

STAGE DEPENDENCIES:
  Stage-1 is a root stage
  Stage-0 is a root stage

STAGE PLANS:
  Stage: Stage-1
    Map Reduce
      Alias -> Map Operator Tree:
        demo 
          TableScan
            alias: demo
            GatherStats: false
            Filter Operator
              isSamplingPred: false
              predicate:
                  expr: (n1 = 'aaa')
                  type: boolean
              Select Operator
                expressions:
                      expr: n1
                      type: string
                      expr: n2
                      type: string
                outputColumnNames: _col0, _col1
                File Output Operator
                  compressed: false
                  GlobalTableId: 0
                  directory: hdfs://localhost:9000/tmp/hive-apache/hive_2013-06-13_19-55-21_578_6086176948010779575/-ext-10001
                  NumFilesPerFileSink: 1
                  Stats Publishing Key Prefix: hdfs://localhost:9000/tmp/hive-apache/hive_2013-06-13_19-55-21_578_6086176948010779575/-ext-10001/
                  table:
                      input format: org.apache.hadoop.mapred.TextInputFormat
                      output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
                      properties:
                        columns _col0,_col1
                        columns.types string:string
                        escape.delim \
                        serialization.format 1
                  TotalFiles: 1
                  GatherStats: false
                  MultiFileSpray: false
      Needs Tagging: false
      Path -> Alias:
        hdfs://localhost:9000/user/hive/warehouse/demo [demo]
      Path -> Partition:
        hdfs://localhost:9000/user/hive/warehouse/demo 
          Partition
            base file name: demo
            input format: org.apache.hadoop.mapred.TextInputFormat
            output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
            properties:
              bucket_count -1
              columns n1,n2
              columns.types string:string
              field.delim ,
              file.inputformat org.apache.hadoop.mapred.TextInputFormat
              file.outputformat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
              location hdfs://localhost:9000/user/hive/warehouse/demo
              name default.demo
              serialization.ddl struct demo { string n1, string n2}
              serialization.format ,
              serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
              transient_lastDdlTime 1370932655
            serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

              input format: org.apache.hadoop.mapred.TextInputFormat
              output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
              properties:
                bucket_count -1
                columns n1,n2
                columns.types string:string
                field.delim ,
                file.inputformat org.apache.hadoop.mapred.TextInputFormat
                file.outputformat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
                location hdfs://localhost:9000/user/hive/warehouse/demo
                name default.demo
                serialization.ddl struct demo { string n1, string n2}
                serialization.format ,
                serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
                transient_lastDdlTime 1370932655
              serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
              name: default.demo
            name: default.demo

  Stage: Stage-0
    Fetch Operator
      limit: -1


Time taken: 5.316 seconds

如何将hive查询转换为抽象语法树？

1 个答案: