在PySpark中将字典另存为CSV / JSON文件

时间:2019-02-20 22:45:00

标签: json csv dictionary pyspark

我有一本字典,其中包含键:文件名值:列和文件名的数据类型。字典看起来像这样:-

{u'file_1.csv': [('WBS_ELEMENT_ID', 'string'),
('WBS_ELEMENT_NAME', 'string'),
('PROJECT_TYPE_ID', 'string'),
('PROJECT_TYPE_NAME', 'string'),
('CONTRACT_ID', 'string'),
('CONTRACT_LINE_NUMBER', 'string'),
('CONTRACT_LINE_NAME', 'string'),
('WBS_FUNC', 'string'),
('WBS_FUNC_DESCR', 'string'),
('WBS_ELMT_STAT', 'string'),
('WBS_ELMT_STAT_DESCR', 'string'),
('ENG_CREATION_DATE', 'string'),
('END_DATE', 'string'),
('PROFIT_CENTER', 'string'),
('COMPANY_CODE', 'string'),
('CONTRACTING_FIRM_CLIENT_ID', 'string'),
('PRODUCT_CODE', 'string')],
u'File_2.csv': [('CONTRACTING_FIRM_CLIENT_ID', 'string'),
('COMPANY_CODE', 'string'),
('PROFIT_CENTER', 'string'),
('FISCAL_MONTH', 'int'),
('CHARGED_HOURS', 'double'),
('FEE_REV_EXTERNAL_CLIENTS', 'double'),
('ENGAGEMENT_MARGIN', 'double'),
('PRODUCT_CODE', 'string'),
('MONTH', 'string'),
('WBS_ELEMENT_ID', 'double')],.......}

我必须将此字典另存为PySpark中的CSV / JSON文件。最好的方法是什么?

0 个答案:

没有答案