Question

根据blog，hdfs使用租约机制来避免两个客户端写入相同的文件。所以我认为不能删除由其他客户端写的文件。但是，这是错的。

当客户端A正在写入lock.txt时，客户端B可以立即删除lock.txt。尽管文件不再存在，客户端A仍可继续写入。就在关闭流时，A会遇到异常： org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease on /user/lock.txt (inode 643845185): File does not exist. Holder DFSClient_NONMAPREDUCE_-1636584585_1 does not have any open files**

为什么会这样？我的haddop版本是2.7.3。

================================

这是我的测试代码：

// write process
object Create {

  private val filePath = "/user/lock.txt"

  def main(args: Array[String]): Unit = {
    println("Create start!")
    val fs = FileSystem.get(new Configuration())
    val os = Option(fs.create(new Path(filePath), false))
    if (os.isDefined) {
      println(s"Create result! $os", System.currentTimeMillis())
      0.until(300).foreach(index => {
        os.get.write(100)
        os.get.flush()
        println("Writing...")
        Thread.sleep(1000)
      })
      println("pre close" + System.currentTimeMillis())
      os.get.close()
      println("close success!" + System.currentTimeMillis())
    }
  }

}

// delete process
object Delete {

  private val filePath = "/user/lock.txt"

  def main(args: Array[String]): Unit = {
    println("Delete start!")
    val fs = FileSystem.get(new Configuration())
    while (!fs.exists(new Path(filePath))) {
      println("File no exist!")
      Thread.sleep(1000)
    }
    println("File exist!")
    while (true) {
      println("try delete!")
      val tmp = Option(fs.delete(new Path(filePath), false))
      if (tmp.isDefined) {
        println(s"delete result:${tmp.get}!" + System.currentTimeMillis())
      }
      println("Try recover")
      if (fs.asInstanceOf[DistributedFileSystem].recoverLease(new Path(filePath))) {
        println("Recover lease success!")
        val res = Option(fs.delete(new Path(filePath), false))
        println(s"File delete success:${res.get}")
      } else {
        println("Recover lease failed!")
      }
      Thread.sleep(1000)
    }
  }

}

hdfs删除正在写入的文件

0 个答案: