我正在尝试从Postgres sql中的元数据表获取列(作为值)的列表
connection = psycopg2.connect(host=ip, user=user, password=password)
cursor = connection.cursor()
cursor.execute("select col_critical from data_quality where col_critical <> 'None' ")
result = cursor.fetchall()
print(result)
readFromDb.criticalFields = ""
for row in result:
print(row)
readFromDb.criticalFields+ = str(row)
之后,我想在pyspark的table的where子句中传递“ readFromDb.criticalFields”的值,例如:
criticalFieldsDf = spark.sql("select * from Raw_table where " + readFromDb.criticalFields+ " is null ")
但是它不起作用,请有人帮忙。