将Pyspark数据帧存储到Hbase

时间:2018-12-05 07:17:55

标签: apache-spark-sql hbase bigdata spark-streaming pyspark-sql

我的代码是将数据存储到hbase中。但是我得到了很多错误。如何定义df,出现“未定义df”错误

catalog = {
    "table": {"namespace":"default", "name":"locdata"},
    "rowkey": "key",
    "columns": {
      "col0": {"cf":"rowkey", "col":"key", "type":"int"},
      "lat":{"cf":"data", "col":"lat", "type":"int"},
      "lon":{"cf":"data", "col":"lon", "type":"int"}
    }
  }

df.write
  .option("catalog",cat)
  .option("newtable","5"),
  .format("org.apache.spark.sql.execution.datasources.hbase"),
  .save(),

0 个答案:

没有答案