Scala:期待尝试[Stream [(String)]]但是给了Try [String]

时间:2018-06-15 13:06:45

标签: scala apache-spark hdfs

我正在尝试阅读文本文件以比较2个文件。我已经编写了代码来读取第一个文件,我期待readFileStream函数给我收集String但我只得到String。 你能看出我弄错了吗?

import java.io.{BufferedReader, FileInputStream, InputStreamReader}
import java.net.URI
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.fs.{FSDataInputStream, FileSystem, Path}
import scala.util.{Failure, Success, Try}

object TestCompareHDFSFiles {
  def main(args: Array[String]): Unit = {
    val hdfs = FileSystem.get(new Configuration())
    val path1 = new Path(args(0))
    val path2 = new Path(args(1))
    readHDFSFile(hdfs, path1, path2)
  }

//  Accept a parameter which implements a close method
    def using[A <: { def close(): Unit }, B](resource: A)(f: A => B): B =
    try {
      f(resource)
    } finally {
      resource.close()
    }

  def readHDFSFile(hdfs: FileSystem, path1: Path, path2: Path): Option[Stream[(String,String)]] = {
        Try(using(new BufferedReader(new InputStreamReader(hdfs.open(path1))))(readFileStream))
        } match {
            case Success(result) => {

            }
            case Failure(ex) => {
              println(s"Could not read file $path1, detail ${ex.getClass.getName}:${ex.getMessage}")
              None
            }
      }

  def readFileStream(br: BufferedReader)= {
      for {
          line <- Try(br.readLine())
          if (line != null )
      } yield line
  }

}

我被震惊了。请帮忙。

谢谢,

0 个答案:

没有答案