Question

我将Spark 1.4.0与Hadoop 2.4.0一起使用，并尝试调用addFile方法，其中输入字符串在x + 1位置有一个空格。我在位置x得到一个错误。关于我做错了什么的建议？

Java exception occurred:
java.net.URISyntaxException: Illegal character in path at index x: /some/path/where/position x+1 is a space character
    at java.net.URI$Parser.fail(Unknown Source)
    at java.net.URI$Parser.checkChars(Unknown Source)
    at java.net.URI$Parser.parseHierarchical(Unknown Source)
    at java.net.URI$Parser.parse(Unknown Source)
    at java.net.URI.<init>(Unknown Source)
    at org.apache.spark.SparkContext.addFile(SparkContext.scala:1316)
    at org.apache.spark.SparkContext.addFile(SparkContext.scala:1303)
    at org.apache.spark.api.java.JavaSparkContext.addFile(JavaSparkContext.scala:671)

修改：代码段和参考

    SparkConf sconf= new SparkConf();
    sconf.setMaster(somename);
    sconf.setAppName(someappname);

    JavaSparkContext jsconf= new JavaSparkContext(sconf);
    jsconf.addFile(dataFile); // dataFile is a String object with contents "/some/path/where/position x+1 is a space character"

addFile参考：https://spark.apache.org/docs/1.4.0/api/java/org/apache/spark/api/java/JavaSparkContext.html#addFile(java.lang.String)

Answer 1

尝试使用：

jsconf.addFile(java.net.URLEncoder.encode(dataFile));

当路径有空格

1 个答案: