联合两个表与火花中不同数量的列

Question

如何合并两个包含不同列数的数据帧A和B，并为数据帧A和B中不常见的列获取空值？

我可以看到spark unionAll不支持此操作并抛出错误

org.apache.spark.sql.AnalysisException: Union can only be performed on tables with the same number of columns, but the left table has 11 columns and the right has 10;

Spark版本：1.6.1

Answer 1

重写您的查询，以便指定所有列并使用null as missing_field来补偿您没有的列

SELECT field1, field2, field3 FROM A UNION ALL SELECT field1, field2, null as field3 FROM B