Question

我对hadoop和hive很新。我正在尝试将数据加载到配置单元表中，我遇到了以下错误。

另一方面，我尝试使用语句stmt.execute("INSERT INTO employee VALUES(1201,'Gopal',45000,'Technical manager')")将记录插入到hive表中它正在成功插入记录，但是在加载大量数据时它失败了。

val filePath=C:\\AllProjects\\xxxxxxx\\src\\main\\resources\\input\\sample.txt
val con = 
DriverManager.getConnection("jdbc:hive2://xxxxxhive.xxxx.com:10000/dehl_dop;principal=hive/xxxxxhive.com.com@internal.xxxxx.com;" +
"mapred.job.queue.name=usa;AuthMech=3;SSL=1;user=zzzz;password=vvvv;" +
"SSLTrustStore=C:\\Program Files\\Java\\jre1.8.0_144\\lib\\security\\hjsecacerts;UseNativeQuery=0")
val stmt = con.createStatement()
print("\n" + "executing the query" +"\n")
stmt.execute(s"load data inpath $filePath into table Employee")

错误

errorMessage:Error while compiling statement: FAILED: ParseException line 1:17 mismatched input 'C' expecting StringLiteral near 'inpath' in load statement), Query: load data inpath C:\xxxxx\xxxxx\xxxxx\xxxxx\xxxxx\xxxxx\sample.txt into table Employee.

任何帮助将不胜感激

Answer 1

add_filter( 'woocommerce_add_cart_item_data', 'add_cart_item_data', 10, 2 ); function add_cart_item_data( $cart_item_data, $product_id ) { //if admin || subscriber $role = get_user_role(); /* i know that isn't the default get user role. i made a shortened version of it in functions.php because I used it a lot in other functions */ if (in_array("admin", $role) || in_array("subscriber", $role)){ // Excluded product categories in this array (Can be IDs, slugs or names) $excl_cats = array( 978 ); $product = wc_get_product( $product_id ); // Excluding product categories if( ! has_term( $excl_cats, 'product_cat', $product_id ) ){ $cart_item_data['ref_price'] = $product->get_price() * 0.85; // Every add to cart action is set as a unique line item $cart_item_data['unique_key'] = md5( microtime().rand() ); } } return $cart_item_data; } add_action( 'woocommerce_before_calculate_totals', 'before_calculate_totals', 10, 1 ); function before_calculate_totals( $cart ) { if ( is_admin() && ! defined( 'DOING_AJAX' ) ) return; if ( did_action( 'woocommerce_before_calculate_totals' ) >= 2 ) return; // Loop through cart items foreach( $cart->get_cart() as $cart_item ) { if( isset( $cart_item['ref_price'] ) { $cart_item['data']->set_price( floatval($cart_item['ref_price']) ); } } }采用字符串文字。

LOAD DATA INPATH需要单引号

$filePath

但是，该命令需要文件位于HDFS上。而你正在阅读你的C盘

stmt.execute(s"load data inpath '$filePath' into table Employee")将读取本地文件系统，但我不确定它是如何工作的，因为它取决于查询实际执行的位置（本地计算机或HiveServer））

我建议你在特定的HDFS位置创建一个外部Hive表，使用必要的模式，然后直接将文本文件复制到HDFS。

以编程方式将文件复制到HDFS是一种选择，但LOAD DATA LOCAL INPATH会更简单。

如果您只想将本地文件加载到HDFS / Hive，Spark会比JDBC更有意义

hadoop fs -put

期望在load语句中'inpath'附近的StringLiteral

1 个答案: