如何在Databricks中使用python将早于y天的文件从存档文件夹移动到某文件夹

时间:2019-01-22 16:34:27

标签: python-3.x databricks

我必须在档案文件夹中找到所有比y天更旧的文件并将它们移动到某个文件夹中。我发现一些档案中比y天更早的文件并尝试移动到其他文件夹。我已经在运行时使用python编写了代码我收到此错误的代码“ java.io.FileNotFoundException:/dbfs/FileStore/Archival/testparquet.parquet”。我已经检查过,文件存在于dbfs中。有人可以帮我吗

from pathlib import Path
import arrow
import os, time, sys
vFilePath="/dbfs/FileStore/"
path = "/dbfs/FileStore/Archival/"
path1="dbfs:/FileStore/Archival/"
#####FOR Dbutils path###
vDbuPath="/FilsStore/Archival/"
deleteFullPath="FileStore/Deleted/"
now = time.time()
print (now)
vdelFullPath=deleteFullPath+"/"
for f in os.listdir(path):
  Filename=str(print(f))
  print(Filename)
  f = os.path.join(path,f)
  print(os.stat(os.path.join(path,f)).st_mtime)
  if os.stat(os.path.join(path,f)).st_mtime < now - 1 * 86400: 
      print("f value: "+f)
      filename=os.path(f)
      print("dbutilspath: " +filename)
      if not os.path.exists("dbfs:/"+deleteFullPath + Filename):         dbutils.fs.mv(filename,"dbfs:/"+deleteFullPath+"testparquet.parquet",recurse=True)

0 个答案:

没有答案