我想使用to_gbq
函数将Pandas数据框上传到Google BigQuery:
import pandas as pd
dfData = pd.DataFrame(llData, columns=lsHeadings)
sProjectID = dBQConfig['sProjectID']
sTargetDataset = dBQConfig['sTargetDataset']
sTargetTable = dBQConfig['sTargetTable']
sTablePath = "{}.{}".format(sTargetDataset, sTargetTable)
dfData.to_gbq(sTablePath, sProjectID, if_exists='replace')
python脚本成功运行,就像这样(在PySpark作业中运行):
Streaming Insert is 100% Complete
16/05/19 16:55:38 INFO SparkContext: Invoking stop() from shutdown hook
Process finished with exit code 0
然后他们的表出现在BigQuery中,然而,它看起来像这样:
它已经差不多24小时了。有什么建议吗?