使用JSON Serde在Hive中加载JSON文件

时间:2019-04-10 16:46:55

标签: json hadoop hive hive-serde

我正在尝试使用json serde在hadoop上上传json文件。我已将jar lib上传到hadoop,但是在运行hive命令时出现错误

我已经将json serde jar文件上传到了/ apps / hive / warehouse / lib路径。现在,当我尝试运行此命令时

ADD JAR /apps/hive/warehouse/lib/json-serde-1.3-jar-with-dependencies.jar;

我收到此错误

  

H110无法提交对帐单。处理语句时出错:   /apps/hive/warehouse/lib/json-serde-1.3.7-SNAPSHOT-jar-with-dependencies.jar   不存在[ERROR_STATUS]

2 个答案:

答案 0 :(得分:0)

尝试在文件名前使用URL(添加 <html> <head> </head> <body> <canvas id="QGL_Chart"></canvas> </body> <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.7.2/Chart.min.js"></script> <script> var dates = ['a', 'b', 'c', 'd', 'e', 'f', 'h']; var open = [1,2,3,4,2,5,1]; var high = [7,4,3,3,3,4,6]; var low = [7,2,2,4,7,6,3]; var close = [9,5,3,4,2,3,4]; var volume = [4,2,1,5,3,6,8]; var ctx = document.getElementById("QGL_Chart").getContext('2d'); var myChart = new Chart(ctx, { type: 'bar', data: data = { labels: [dates[0], dates[1], dates[2], dates[3], dates[4], dates[5], dates[6]], datasets: [ { type: 'line', label: "Open", fill: false, yAxisID: 'y-axis-a', lineTension: 0.1, backgroundColor: 'rgb(75, 214, 238)', borderColor: 'rgb(75, 214, 238)', borderCapStyle: 'butt', borderDash: [], borderDashOffset: 0.0, borderJoinStyle: 'miter', pointBorderColor: 'rgb(75, 214, 238)', pointBackgroundColor: 'rgb(75, 214, 238)', pointBorderWidth: 1, pointHoverRadius: 4, pointHoverBackgroundColor: 'rgb(75, 214, 238)', pointHoverBorderColor: 'rgb(75, 214, 238)', pointHoverBorderWidth: 3, pointRadius: 5, pointHitRadius: 10, data: [open[0], open[1], open[2], open[3], open[4], open[5], open[6]], }, { type: 'line', label: "High", fill: false, yAxisID: 'y-axis-a', lineTension: 0.1, backgroundColor: 'rgb(210, 221, 72)', borderColor: 'rgb(210, 221, 72)', borderCapStyle: 'butt', borderDash: [], borderDashOffset: 0.0, borderJoinStyle: 'miter', pointBorderColor: 'rgb(210, 221, 72)', pointBackgroundColor: 'rgb(210, 221, 72)', pointBorderWidth: 1, pointHoverRadius: 4, pointHoverBackgroundColor: 'rgb(210, 221, 72)', pointHoverBorderColor: 'rgb(210, 221, 72)', pointHoverBorderWidth: 3, pointRadius: 5, pointHitRadius: 10, data: [high[0], high[1], high[2], high[3], high[4], high[5], high[6]], }, { type: 'line', label: "Low", fill: false, yAxisID: 'y-axis-a', lineTension: 0.1, backgroundColor: 'rgb(238, 79, 75)', borderColor: 'rgb(238, 79, 75)', borderCapStyle: 'butt', borderDash: [], borderDashOffset: 0.0, borderJoinStyle: 'miter', pointBorderColor: 'rgb(238, 79, 75)', pointBackgroundColor: 'rgb(238, 79, 75)', pointBorderWidth: 1, pointHoverRadius: 4, pointHoverBackgroundColor: 'rgb(238, 79, 75)', pointHoverBorderColor: 'rgb(238, 79, 75)', pointHoverBorderWidth: 3, pointRadius: 5, pointHitRadius: 10, data: [low[0], low[1], low[2], low[3], low[4], low[5], low[6]], }, { type: 'line', label: "Close", fill: false, yAxisID: 'y-axis-a', lineTension: 0.1, backgroundColor: 'rgb(28, 175, 154)', borderColor: 'rgb(28, 175, 154)', borderCapStyle: 'butt', borderDash: [], borderDashOffset: 0.0, borderJoinStyle: 'miter', pointBorderColor: 'rgb(28, 175, 154)', pointBackgroundColor: 'rgb(28, 175, 154)', pointBorderWidth: 1, pointHoverRadius: 4, pointHoverBackgroundColor: 'rgb(28, 175, 154)', pointHoverBorderColor: 'rgb(28, 175, 154)', pointHoverBorderWidth: 3, pointRadius: 5, pointHitRadius: 10, data: [close[0], close[1], close[2], close[3], close[4], close[5], close[6]], }, { label: 'Volume', //1D2939 yAxisID: 'y-axis-b', data: [volume[0], volume[1], volume[2], volume[3], volume[4], volume[5], volume[6]], barPercentage: '1', categoryPercentage: '1', backgroundColor: 'rgb(29, 41, 57)', borderColor: 'rgb(29, 41, 57)', borderWidth: '1', borderSkipped: 'bottom', hoverBackgroundColor: 'rgb(29, 41, 57)', hoverBorderColor: 'rgb(29, 41, 57)', hoverBorderWidth: '3', }, ] }, options: { title: { display: true, text: 'Share Price - Past 7 Days', fontSize: '20', fontFamily: 'Open Sans, sans-serif', // fontColor // fontStyle // padding // lineHeight }, scales: { xAxes: [{ ticks: { min: 0 } }], yAxes: [{ position: "left", id: "y-axis-a", }, { position: "right", id: "y-axis-b", }] } } }); </script> </html>

file//

如果您的Hive为repository及更高版本,那么您还应该能够从1.2.0添加jar。

答案 1 :(得分:0)

好像您的jar位于HDFS位置。使用:

add jar hdfs:///apps/hive/warehouse/lib/json-serde-1.3-jar-with-dependencies.jar;