当Spark独立群集不可用时,Spark驱动程序会抛出奇怪的异常

时间:2015-12-09 12:05:56

标签: java apache-spark

Spark版本:1.4.1

我在驱动程序下运行并尝试为远程spark独立群集创建Spark Context。当火花报告错误后,远程火花簇不可用。例外的原因是什么(

 <div style="text-align: center;">
  	<h1>Title</h1>
	  <h2>tagline</h2>
	<div id="container">
      <!-- Module1 -->
      <div class="module" style="background-color: red;">
        <table>
          <tbody>
            <tr>
              <td valign="top">
                <p><strong>Module 1</strong></p>
                <p>lorem ipsum</p>
                <p>lorem ipsum</p>
                <p>lorem ipsum</p>
              </td>
            </tr>
            <tr>
              <td valign="bottom">
                <div id="mc_embed_signup">
                  <form>
                    <div>
                      <div><input name="EMAIL" type="email" value="" placeholder="email address" /></div>
                        
                     </div>
                    
                      <div><input type="submit" value="button" /></div>
                    </div>
                  </form>
                </div>
              </td>
            </tr>
      
          </tbody>
        </table>
      </div>
      <!-- Fin module 1, début module 2 -->
      <div class="module" style="background-color: green;">
        <table>
          <tbody>
            <tr>
              <td valign="top">
                <p><strong>Module 2</strong></p>
              </td>
            </tr>
            <tr>
              <td valign="bottom">
                <div>
                  <form>
                    <div>
                      <div><input name="EMAIL" type="email" placeholder="email address" /></div>
                        
                     </div>
                    
                      <div class="clear"><input name="subscribe" type="submit" value="button" /></div>
                    </div>
                  </form>
                </div>
              </td>
            </tr>
       
          </tbody>
        </table>
      </div>
<!-- Fin module 2, début module 3 -->
      <div class="module" style="background-color: yellow;">
        <table>
          <tbody>
            <tr>
              <td valign="top">
                <p><strong>Module 3</strong></p>
                <p>lorem ipsum</p>
              </td>
            </tr>
            <tr>
              <td valign="bottom">
                <div>
                  <form>
                    <div>
                      <div ><input name="EMAIL" type="email" placeholder="email address" /></div>
                        
                     </div>
                    
                      <div class="clear"><input name="subscribe" type="submit" value="button" /></div>
                    </div>
                  </form>
                </div>
              </td>
            </tr>
            
          </tbody>
        </table>
      </div>
      <!-- Fin module 3 -->
    </div>
</div>

ERROR OneForOneStrategy: 
java.lang.NullPointerException

)以及如何消除它们?

注意:当我使用运行本地或远程独立群集时,我的应用程序运行正常。

Java程序

ERROR SparkContext: Error initializing SparkContext.
java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext

Spark日志

public class SparkContextTest {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        JavaSparkContext sc = null;
        try {
            SparkConf conf = new SparkConf();
        conf.setAppName("testABC");
        conf.set("spark.scheduler.mode", "FAIR");

        conf.setMaster("spark://remote-server:7077")
                .set("spark.driver.allowMultipleContexts", "false")
                .set("spark.executor.memory", "1g")
                .set("spark.driver.maxResultSize", "1g");

         sc = new JavaSparkContext(conf);
        } catch(Exception e){                
            e.printStackTrace();
        } finally {
            if(null != sc){
                sc.clearCallSite();
                sc.close();
            }
        }

    }

}

0 个答案:

没有答案