Question

我正在尝试在Spark集群环境中部署一个火花流式在线wordcount程序。

import java.util.Arrays;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.function.FlatMapFunction;
import org.apache.spark.api.java.function.Function2;
import org.apache.spark.api.java.function.PairFunction;
import org.apache.spark.streaming.Durations;
import org.apache.spark.streaming.api.java.JavaDStream;
import org.apache.spark.streaming.api.java.JavaPairDStream;
import org.apache.spark.streaming.api.java.JavaReceiverInputDStream;
import org.apache.spark.streaming.api.java.JavaStreamingContext;
import scala.Tuple2;

public class WordCountOnline {
    public  static void main(String[] args){
    SparkConf conf =new SparkConf();
   conf.setMaster("spark://SparkMaster:7077").setAppName("WordCountOnline");
    JavaSparkContext sc=new JavaSparkContext(conf);
    JavaStreamingContext jsc=new JavaStreamingContext(conf,Durations.seconds(5));
    JavaReceiverInputDStream lines=jsc.socketTextStream("SparkMaster",9999); 
    JavaDStream<String> words=lines.flatMap(new FlatMapFunction<String, String>() {
    public Iterable<String> call(String line) throws Exception {
        return Arrays.asList(line.split(" "));
    }
});
    JavaPairDStream<String,Integer> pairs=words.mapToPair(new PairFunction<String, String, Integer>() {
    public Tuple2<String, Integer> call(String word) throws Exception {
          return new Tuple2<String,Integer>(word,1);        
    }
});
    JavaPairDStream<String, Integer> wordsCount=pairs.reduceByKey(new Function2<Integer, Integer, Integer>() {

    public Integer call(Integer v1, Integer v2) throws Exception {
        // TODO Auto-generated method stub
        return v1+v2;
    }
});
   wordsCount.print();
   jsc.start();
   jsc.awaitTermination();
   jsc.close();
}
}

`我正在部署火花集群环境。

$SPARK_HOME$/bin# ./spark-submit --class sparkstreaming.WordCountOnline --master spark://SparkMaster:7077 /root/Documents/WordCountOnline.jar

并且存在问题

FAILED SelectChannelConnector@0.0.0.0:4040: java.net.BindException: Address already in use java.net.BindException: Address already in use

我已经通过

检查了端口4040

 netstat  -apn | grep 4040

结果显示端口4040没有被监听，此外，我在代码中选择的端口是9090.为什么会出现这种问题？

FAILED SelectChannelConnector@0.0.0.0:4040:java.net.BindException：地址已经在使用java.net.BindException：地址已在使用中

0 个答案: