Spark中是否有类似的API与twitter.scalding.addTrap一起处理异常

Question

我正在将一个项目从hadoop移植到Spark 2.1.0。以前，它使用twitter.scalding.addTrap来处理异常，例如： https://github.com/scalding-io/ProgrammingWithScalding/blob/master/chapter3/src/main/scala/addTrap.scala

对于Spark，我使用sc.textFile（InputPath）读取输入，但我不知道如何执行以前的异常处理函数。

Answer 1

您可以使用Try。

import scala.io.StdIn
import scala.util.{Try, Success, Failure}

def divide: Try[Int] = {
    val dividend = Try(StdIn.readLine("Enter an Int that you'd like to divide:\n").toInt)
    val divisor = Try(StdIn.readLine("Enter an Int that you'd like to divide by:\n").toInt)
    val problem = dividend.flatMap(x => divisor.map(y => x/y))
    problem match {
        case Success(v) =>
            println("Result of " + dividend.get + "/"+ divisor.get +" is: " + v)
            Success(v)
        case Failure(e) =>
            println("You must've divided by zero or entered something that's not an Int. Try again!")
            println("Info from the exception: " + e.getMessage)
            divide
    }
}