如何在PySpark中将字典转换为数据框

Question

我有输入

Error occurred for request GET /artifactory/api/system/version 
HTTP/1.1: sun.security.validator.ValidatorException: PKIX path building 
failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target.

我需要的输出

{'A':'1','B':'1'}

我尝试过的

代码如下。这不起作用

+----+----+
|A   |B   |
+----+----+
|   1|   2|
+----+----+

Answer 1

我找到了使用json解析器的简单方法。

    spark = SparkSession.builder.appName('abc').enableHiveSupport().getOrCreate()
    sc = spark.sparkContext
    dict_lst = {'A': '1','B':'2'}

    rdd = sc.parallelize([dict_lst])
    print(type(rdd))
    df = spark.read.json(rdd)

    df.show()