Question

我的Scala计划：

import org.apache.spark._
import org.apache.spark.SparkContext._

object WordCount {
    def main(args: Array[String]) {
      val inputFile = args(0)
      val outputFile = args(1)
      val conf = new SparkConf().setAppName("wordCount")
      // Create a Scala Spark Context.
      val sc = new SparkContext(conf)
      // Load our input data.
      val input =  sc.textFile(inputFile)
      // Split up into words.
      val words = input.flatMap(line => line.split(" "))
      // Transform into word and count.
      val counts = words.map(word => (word, 1)).reduceByKey{case (x, y) => x + y}
      // Save the word count back out to a text file, causing evaluation.
      counts.saveAsTextFile(outputFile)
    }
}

我收到错误：

Error: Could not find or load main class WordCount

Answer 1

您尚未设置主人的属性。

val conf = new SparkConf().setAppName("wordCount").setMaster(local[*])

Answer 2

我认为可能是类路径的东西，没有正确设置。

如果您正在使用Intellij，请将该目录标记为源根目录，即可。

我在scala中运行Spark WordCount时遇到错误

2 个答案: