火花作业结果最终状态未定义

时间:2015-12-31 06:57:11

标签: apache-spark

在Spark上安装一个单独的节点后,在同一个盒子上我安装了Ambari服务器。我尝试用Pi示例验证一个spark-job: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_spark-quickstart/content/run_spark_pi.html

我试过的步骤

  1. @Override protected void onDestroy() { super.onDestroy(); removeActivityFromTransitionManager(Activity activity); } private static void removeActivityFromTransitionManager(Activity activity) { if (Build.VERSION.SDK_INT < 21) { return; } Class transitionManagerClass = TransitionManager.class; try { Field runningTransitionsField = transitionManagerClass.getDeclaredField("sRunningTransitions"); runningTransitionsField.setAccessible(true); //noinspection unchecked ThreadLocal<WeakReference<ArrayMap<ViewGroup, ArrayList<Transition>>>> runningTransitions = (ThreadLocal<WeakReference<ArrayMap<ViewGroup, ArrayList<Transition>>>>) runningTransitionsField.get(transitionManagerClass); if (runningTransitions.get() == null || runningTransitions.get().get() == null) { return; } ArrayMap map = runningTransitions.get().get(); View decorView = activity.getWindow().getDecorView(); if (map.containsKey(decorView)) { map.remove(decorView); } } catch (NoSuchFieldException e) { e.printStackTrace(); } catch (IllegalAccessException e) { e.printStackTrace(); } }
  2. su hdfs
  3. `./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn-cluster --num-executors 1 lib / spark-examples * .jar 10

    [注意:由于我只有一个数据节点在运行,因此我已将激励器从3改为1)

  4. 我应该看到:

    cd /usr/hdp/current/spark-client

    但遇到问题,我看到以下输出:

    client token: N/A
    diagnostics: N/A
    ApplicationMaster host: N/A
    ApplicationMaster RPC port: 0
    queue: default
    start time: 1428686924325
    final status: SUCCEEDED
    
    Log Type: stdout
    Log Upload Time: 10-Jun-2015 17:13:33
    Log Length: 23
    Pi is roughly 3.142532
    

    我也尝试了不同的Spark工作。但是,所有人都处于同样的未定义状态。

    我缺少什么?感谢你的帮助。

0 个答案:

没有答案