Java Spark withColumn绑定属性错误

时间:2019-08-19 09:15:40

标签: apache-spark

在Java中调用spark udf时发生“ TreeNodeException:绑定属性,树:visit_datetime#161”错误。

火花版本:'2.3.0' Java版本:“ 1.8.0”

java伪代码

udfs.put(function.name + "3", udf((String a, Boolean b, Boolean c) -> "hello", DataTypes.StringType));
udfs.put(function.name + "5", udf((String a, String b, String c, Boolean d, Boolean e) -> "world", DataTypes.StringType));

UserDefinedFunction udf = (UserDefinedFunction)udfs.get(function.name + seq.length());
if (udf != null) {
   tableDataSet = tableDataSet.withColumn(column + "_", udf.apply(seq));
}

tableDataSet.explain(true);
tableDataSet.show(5);
错误消息:

原因:java.lang.RuntimeException:在[门诊病人​​编号#0,门诊病人编号#1,病人ID#2,医院住院病人编号#3,门诊病人visit_count#4,cdt_visit_doctor#5,cdt_visit_doctor_ment#6,找不到门诊日期时间#161 #7,cdt_visit_dept_speciality#8,cdt_visit_dept_code#9,rstd_visit_dept_name#10,rstd_visit_dept_code#11,rstd_visit_speciality#12,cdt_visit_type#13,visit_datetime#14,诊所,registered_dept#20,registered_dept_code#21,registered_speciality#22,rstd_regist_dept_name#23,rstd_regist_dept_code#24,rstd_regist_speciality#25,selected_doctor#26,selected_doctor_id#27,time_of_register ### 28,triage_tiage_sage#29,triage_id #32,yidu_register_status#33,yidu_visit_type#34,yidu_data_credibility#35,first_or_further_visit#36,visit_dept_code#37,clinic_type#38]

==身体计划==

!项目[修剪(outdoor_visit_id#0,无)AS outpatient_visit_id#361,修剪(outpatient_no#1,无)AS outpatient_no#401,修剪(Patient_id#2,无)AS Patient_id#441,修剪(dom_hospital_no#3,无) AS dom_hospital_no#481,修剪(门诊次数#4,无)AS门诊_访客数#521,修剪(cdt_visit_doctor#5,无)AS cdt_visit_doctor#561,修剪(cdt_visit_doctor_id#6,无)AS cdt_visit_doctor_id#6,无无)AS cdt_visit_department#641,修剪(cdt_visit_dept_speciality#8,无)AS cdt_visit_dept_speciality#681,trim(cdt_visit_dept_code#9,无)AS cdt_visit_dept_code#721,trim(rstd_visit_dept_dept_name,trim(rstd_visit_dit_dept_name 11,无)AS rstd_visit_dept_code#801,trim(rstd_visit_speciality#12,None)AS rstd_visit_speciality#841,trim(cdt_visit_type#13,None)AS cdt_visit_type#881,trim(UDF(​​visit_datetime#14,false,true),None) AS visit_datetime#921,修剪(clinic_document_id#15,无)AS clinic_document_id#961,修剪(UDF(​​pat ient_age#16,visit_datetime#161,birth_date#201,false,true),无)AS Patient_age#1001,trim(cdt_fee_type#17,None)AS cdt_fee_type#1041,trim(cdt_register_status#18,None)AS cdt_register_status#1081, trim(UDF(​​出生日期#19,false,true),无)AS出生日期#1121,trim(registered_dept#20,None)AS Registered_dept#1161,trim(registered_dept_code#21,None)ASregistered_dept_code#1201,trim(registered_speciality# 22,无)作为AS Registerd_speciality#1241,trim(rstd_regist_dept_name#23,无)作为AS rstd_regist_dept_name#1281,...还有15个字段]

0 个答案:

没有答案