Question

我几周来一直在与这个错误作斗争。我尝试了许多不同的应用程序，但无法找到与此错误的任何一致性。有时，如果我更改作业书签设置，它会暂停，启用或返回禁用状态，它会开始工作。我有一个java jar，我正在引用胶水作业，我从中调用了一些方法。有时，如果我重建工件，则作业开始工作，不再抛出此错误。我有另一个工作使用相同的jar并且不会抛出错误。我尝试创建一个新工作重新开始，但我看到同样的问题。这是日志中提供的错误堆栈。应用程序中的代码是从S3读取的简单数据帧，并写入s3中的另一个位置。

val df = spark.read.parquet(source)

df.write.mode("overwrite").format("parquet").save(destination)

错误堆栈：

18/04/30 14:40:35 ERROR ApplicationMaster: User class threw exception: java.lang.NoSuchMethodError: com.amazonaws.services.glue.AWSGlue.getJobBookmark(Lcom/amazonaws/services/glue/model/GetJobBookmarkRequest;)Lcom/amazonaws/services/glue/model/GetJobBookmarkResult;
java.lang.NoSuchMethodError: com.amazonaws.services.glue.AWSGlue.getJobBookmark(Lcom/amazonaws/services/glue/model/GetJobBookmarkRequest;)Lcom/amazonaws/services/glue/model/GetJobBookmarkResult;
at com.amazonaws.services.glue.util.JobBookmarkServiceShim$$anonfun$2.apply(JobBookmarkUtils.scala:54)
at com.amazonaws.services.glue.util.JobBookmarkServiceShim$$anonfun$2.apply(JobBookmarkUtils.scala:54)
at scala.util.Try$.apply(Try.scala:192)
at com.amazonaws.services.glue.util.JobBookmarkServiceShim.<init>(JobBookmarkUtils.scala:54)
at com.amazonaws.services.glue.util.JobBookmark$.configure(JobBookmarkUtils.scala:178)
at com.amazonaws.services.glue.util.Job$.init(Job.scala:68)
at com.amazonaws.services.glue.util.Job$.init(Job.scala:32)
at NetezzaRawToRefined$.main(script_2018-04-30-14-39-54.scala:16)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.amazonaws.services.glue.util.GlueExceptionWrapper$$anonfun$1.apply$mcV$sp(GlueExceptionWrapper.scala:29)
at com.amazonaws.services.glue.util.GlueExceptionWrapper$$anonfun$1.apply(GlueExceptionWrapper.scala:29)
at com.amazonaws.services.glue.util.GlueExceptionWrapper$$anonfun$1.apply(GlueExceptionWrapper.scala:29)
at scala.util.Try$.apply(Try.scala:192)
at com.amazonaws.services.glue.util.GlueExceptionWrapper$.delayedEndpoint$com$amazonaws$services$glue$util$GlueExceptionWrapper$1(GlueExceptionWrapper.scala:28)
at com.amazonaws.services.glue.util.GlueExceptionWrapper$delayedInit$body.apply(GlueExceptionWrapper.scala:11)
at scala.Function0$
class.apply$mcV$sp(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:392)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at com.amazonaws.services.glue.util.GlueExceptionWrapper$.main(GlueExceptionWrapper.scala:11)
at com.amazonaws.services.glue.util.GlueExceptionWrapper.main(GlueExceptionWrapper.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:635)
18/04/30 14:40:35 INFO ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.lang.NoSuchMethodError: com.amazonaws.services.glue.AWSGlue.getJobBookmark(Lcom/amazonaws/services/glue/model/GetJobBookmarkRequest;)Lcom/amazonaws/services/glue/model/GetJobBookmarkResult;)

Answer 1

public class AdapterDemo extends RecyclerView.Adapter<AdapterDemo.MyViewHolder> {

    private ArrayList<String> list = new ArrayList<>();
    private Context context;

    class MyViewHolder extends RecyclerView.ViewHolder {

        MyViewHolder(View view) {
            super(view);
            view.setOnClickListener(new View.OnClickListener() {
                @Override
                public void onClick(View v) {
                    Intent intent = new Intent(context, YourActivity.class);
                    String s = list.get(getAdapterPosition()); // here you will get current item
                    intent.putExtra("key", s);
                    context.startActivity(intent);
                }
            });
        }
    }



    public AdapterDemo(Context context) {
        this.context = context;
    }

    @Override
    public MyViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
        View itemView = LayoutInflater.from(parent.getContext())
                .inflate(R.layout.layout_nothing_to_display, parent, false);

        return new MyViewHolder(itemView);
    }

    @Override
    public void onBindViewHolder(MyViewHolder holder, int position) {
        String s = list.get(position);
        // TODO: 4/29/2018  your code here
    }

    @Override
    public int getItemCount() {
        if (list == null) list = new ArrayList<>();
        return list.size();
    }


    public void insertItemInList(String beanChat) {
        if (list == null) list = new ArrayList<>();
        list.add(beanChat);
        notifyItemInserted(list.size() - 1);
    }


    public void insertItemsInList(ArrayList<String> myList) {
        if (list == null) list = new ArrayList<>();
        for (String beanChat : myList) {
            insertItemInList(beanChat);
        }
    }

    public ArrayList<String> getList() {
        if (list == null) list = new ArrayList<>();
        return list;
    }

    public void clearList() {
        if (list == null) return;
        list.clear();
        notifyDataSetChanged();
    }
}

如果运行时的jar版本与打包版本不同，则通常会发生此错误。我建议你确保你装的是相同版本的jar。

AWS Glue：命令失败，错误代码为1

1 个答案: