Executing "Select *" query for multiple tables using Apache NiFi

Question

Requirement: From an UI I am getting selected list of tables of a database. Data from these tables are to extracted and stored in file location. We are expected to use NiFi Rest APIs as there is a requirement for custom UI. So we are invoking NiFi processors using REST APIs.

Issue: With ExecuteSQL processor I can execute one SQL query at a time. Since the query is same for all of the tables (select * ...) I can pass in the table name as an attribute to the processor. But I am getting the table names from a REST API call so its delimited string.

the issue is how to invoke the ExecuteSQL in a loop for each table name received in a delimited String.

Kindly let me know if additional information is required.

Answer 1

ExecuteSQL无法在单个流文件上循环。相反，您可以使用ReplaceText将分隔的字符串放入流文件的主体中，然后使用SplitText在分隔符上拆分。您将获得多个流文件，每个文件的主体包含一个字段。然后，您可以再次使用ReplaceText来匹配整个文本，并替换为该组周围的SELECT语句：

DIV

然后您可以将所有流文件发送到ExecuteSQL，它将一次执行每个SELECT文件（如果您设置多个并发任务，则同时执行。