Question

我正在使用spark 2.3，并将sparkThrift与beeline连接起来。

Hive jdbc版本1.2.1 Spark SQL版本2.3.1

我正在尝试使用skip header属性创建外部表，但是select命令总是返回数据以header为第一行，这是我的create查询

CREATE EXTERNAL TABLE datasourcename11(
`retail_invoice_detail_sys_invoice_no` STRING,
`store_id` STRING,
`retail_invoice_detail_invoice_time` STRING,
`retail_invoice_detail_invoice_date` string,
`cust_id` STRING,
`article_code` INTEGER,
`retail_invoice_detail_base_price` INTEGER,
`retail_invoice_detail_sale_price` INTEGER,
`retail_invoice_detail_quantity` DOUBLE,
`retail_invoice_detail_total_amount` DOUBLE
) 
ROW FORMAT DELIMITED  FIELDS TERMINATED BY ',' 
LINES TERMINATED BY '\n'  
LOCATION '/home/java_services/backend/demo/' 
TBLPROPERTIES('skip.header.line.count'=1);

Answer 1

此属性skip.header.line.count=1仅在Hive中受支持。

解决方法是使用过滤器

retail_invoice_detail_sys_invoice_no!=<col name in header>

TBLPROPERTIES（'skip.header.line.count'='1'）在通过蜂巢与配置单元jdbc 1.2.1连接的sparkThrift上不起作用

1 个答案: