Nifi-如何将数据放入Hive数据库？

Question

我正在构建Nifi流以从kafka中获取json元素并将其写入Have表中。

但是，关于处理器及其使用方法的文档很少甚至没有。

我打算做的是以下事情：

kafka consume --> ReplaceText --> PutHiveQL

使用kafka主题做得很好。我收到一个json字符串。

我想提取json数据（带有replaceText）并将其放入配置单元表（PutHiveQL）中。

但是，我绝对不知道该怎么做。文档没有帮助，也没有确切的处理器使用示例（或者我找不到一个示例）。

• 我的理论解有效吗？
• 如何提取json数据，构建HQL查询并将其发送到我的本地配置单元数据库？

Answer 1

基本上，您希望将记录从kafka转换为HQL请求，然后将请求发送到putHiveQl处理器。

我不认为转换kafka记录-> putHQL可以通过替换文本来完成（缝线有点难/棘手）。通常，我使用自定义的groovy脚本处理器来执行此操作。

---

修改

全局概述：

EvaluateJsonPath

这将提取我的Json流文件的属性replacement value和import UIKit class ViewController: UIViewController { var ht = -90 var ww = 80 var hw = 80 var arrTextFields = [UIImageView]() var b7 = UIButton() override func viewDidLoad() { [b7].forEach { $0.translatesAutoresizingMaskIntoConstraints = false view.addSubview($0) $0.backgroundColor = .systemOrange } b7.frame = CGRect(x: view.center.x-115, y: view.center.y + 200, width: 70, height: 40) b7.addTarget(self, action: #selector(addBOx), for: .touchUpInside) for view in self.arrTextFields { view.isUserInteractionEnabled = true view.addGestureRecognizer(UITapGestureRecognizer(target: self, action: #selector(viewClicked))) } } @objc func viewClicked(_ recognizer: UITapGestureRecognizer) { print("tap") } //func that adds imageview. @objc func addBOx() { let subview = UIImageView() subview.isUserInteractionEnabled = true arrTextFields.append(subview) view.addSubview(subview) subview.frame = CGRect(x: view.bounds.midX - 0, y: view.bounds.midY + CGFloat(ht), width: CGFloat(ww), height: 35) subview.backgroundColor = .purple ht += 50 arrTextFields.append(subview) } } ，并将它们作为流文件的属性。

替换文字

此操作将流文件字符串设置为空字符串，并用http.js:147 POST https://www.sandbox.paypal.com/v1/payment-experience/web-profiles 401 (Unauthorized) types.js:121 Uncaught Error: Request to post https://www.sandbox.paypal.com/v1/payment-experience/web-profiles failed with 401 error. Correlation id: bfca9a9c3fdfc { "name": "AUTHENTICATION_FAILURE", "debug_id": "bfca9a9c3fdfc", "message": "Authentication failed due to invalid authentication credentials or a missing Authorization header", "information_link": "https://developer.paypal.com/docs/api/payment-experience/#errors", "details": [] } at XMLHttpRequest.xhrLoad (http.js:114) at Object._RECEIVE_MESSAGE_TYPE.<computed> [as postrobot_message_response] (types.js:121) at receiveMessage (index.js:114) at messageListener (index.js:140) at Object._RECEIVE_MESSAGE_TYPE.<computed> [as postrobot_message_response] (types.js:121) at receiveMessage (index.js:114) at messageListener (index.js:140) _RECEIVE_MESSAGE_TYPE.<computed> @ types.js:121 receiveMessage @ index.js:114 messageListener @ index.js:140 setTimeout (async) dispatchPossiblyUnhandledError @ exceptions.js:16 (anonymous) @ promise.js:122 setTimeout (async) reject @ promise.js:120 dispatch @ promise.js:179 reject @ promise.js:127 dispatch @ promise.js:179 reject @ promise.js:127 dispatch @ promise.js:186 reject @ promise.js:127 (anonymous) @ promise.js:157 dispatch @ promise.js:184 reject @ promise.js:127 (anonymous) @ promise.js:51 respond @ client.js:147 _RECEIVE_MESSAGE_TYPE.<computed> @ types.js:126 receiveMessage @ index.js:114 messageListener @ index.js:140 serialize.js:175 Uncaught Error: Error: Request to post https://www.sandbox.paypal.com/v1/payment-experience/web-profiles failed with 401 error. Correlation id: bfca9a9c3fdfc { "name": "AUTHENTICATION_FAILURE", "debug_id": "bfca9a9c3fdfc", "message": "Authentication failed due to invalid authentication credentials or a missing Authorization header", "information_link": "https://developer.paypal.com/docs/api/payment-experience/#errors", "details": [] } at XMLHttpRequest.xhrLoad (http.js:114) at Object._RECEIVE_MESSAGE_TYPE.<computed> [as postrobot_message_response] (types.js:121) at receiveMessage (index.js:114) at messageListener (index.js:140) at Object._RECEIVE_MESSAGE_TYPE.<computed> [as postrobot_message_response] (types.js:121) at receiveMessage (index.js:114) at messageListener (index.js:140) at deserializeError (serialize.js:175) at serialize.js:212 at util.js:140 at eachArray (util.js:102) at each (util.js:116) at replaceObject (util.js:138) at util.js:147 at eachObject (util.js:109) at each (util.js:118) at replaceObject (util.js:138) 属性代替，我在其中构建查询。

Answer 2

您可以使用Puthivestreaming过程直接注入流数据。 创建一个具有与流匹配的结构的ORC表，并将流传递给它起作用的PUTHIVE3STreaming处理器。