Bash to Python:展平目录树

时间:2012-10-08 11:51:25

标签: python bash

在类Unix系统上我使用这个脚本,我想在移植到Python以便在Windows主机上执行时提供一些帮助:


#!/bin/bash

SENTINEL_FILENAME='__sentinel__'
SENTINEL_MD5_CHECKSUM=''
SENTINEL_SHA_CHECKSUM=''

function is_directory_to_be_flattened() {

  local -r directory_to_consider="$1"
  local -r sentinel_filepath="${directory_to_consider}/${SENTINEL_FILENAME}"

  if [ ! -f "${sentinel_filepath}" ]; then
    return 1
  fi

  if [[
      "$(
         md5 "${sentinel_filepath}" \
           | awk '{ print $NF }' 2> /dev/null
       )" \
        == "${SENTINEL_MD5_CHECKSUM}"
    && \
      "$(
         shasum -a 512 "${sentinel_filepath}" \
           | awk '{ print $1 }' 2> /dev/null
       )" \
        == "${SENTINEL_SHA_CHECKSUM}"
  ]]; then
    return 0
  else
    return 1
  fi
}

function conditionally_flatten() {

  local -r directory_to_flatten="$1"
  local -r flatten_into_directory="$2"

  if is_directory_to_be_flattened "${directory_to_flatten}"; then

    if [ ! -d "${flatten_into_directory}" ]; then
      mkdir -v "${flatten_into_directory}"
    fi

    for file_to_move in $(find ${directory_to_flatten} -type f -maxdepth 1); do
      mv \
        -v \
        -n \
        "${file_to_move}" \
        "${flatten_into_directory}"
    done
  fi
}

function flatten_directory() {

  local -r directory_to_flatten="$1"
  local -r descend_depth="$2"

  local -r flattened_directory="${directory_to_flatten}/__flattened__"

  if [ ! -d "${directory_to_flatten}" ]; then
    printf "The argument '%s' does not seem to be a directory.\n" \
      "${directory_to_flatten}" \
      >&2
    return
  fi

  find "${directory_to_flatten}" \
    -type d \
    -maxdepth "${descend_depth}" \
  | \
    while read directory_path; do
      conditionally_flatten \
        "${directory_path}" \
        "${flattened_directory}"
    done
}

n_arguments="$#"

if [ "${n_arguments}" -eq 1 ]; then
  flatten_directory "$1" '1' # maybe use a constant, not a "magic #" here?
else
  echo usage: "$0" /path/to/directory/to/flatten
fi

unset is_directory_to_be_flattened
unset conditionally_flatten
unset flatten_directory

你如何将它移植到Win Python?我是Python和Bash脚本的初学者..

如果您发现它实际上没有任何方式,请随意升级我的实现,请说明理由。这不是“代码审查”,而是我在Bash工作中的“竖起大拇指/竖起大拇指”会让我感觉到我是在改进还是应该改变我完全学习的方式......


我们接下来,我在Python中的尝试:(如果需要的话,批评它 hard ,这是我学习的唯一方法!)


#!/usr/bin/env python2.7

import sys
import os
import shutil

SENTINEL_FILENAME=''
SENTINEL_MD5_CHECKSUM=''
SENTINEL_SHA_CHECKSUM=''

DEFAULT_DEPTH = 1
FLATTED_DIRECTORY_NAME = '__flattened__'

def is_directory_to_be_flattened(directory_to_consider):
  sentinel_location = os.path.join(directory_to_consider, SENTINEL_FILENAME)
  if not os.path.isfile(sentinel_location):
    return False
  import hashlib
  with open(sentinel_location) as sentinel_file:
    file_contents = sentinel_file.read()
    return (hashlib.md5(file_contents).hexdigest() == SENTINEL_MD5_CHECKSUM
      and hashlib.sha512(file_contents).hexdigest() == SENTINEL_SHA_CHECKSUM)

def flatten(directory, depth, to_directory, do_files_here):
  if depth < 0:
    return
  contained_filenames = [f for f in os.listdir(directory)]
  if do_files_here:
    for filename in contained_filenames:
      if filename == SENTINEL_FILENAME:
        continue
      filepath = os.path.join(directory, filename)
      if not os.path.isfile(filepath):
        continue
      file_to = os.path.join(to_directory, filename)
      if not os.path.isdir(to_directory):
        os.makedirs(to_directory)
      if not os.path.isfile(file_to):
        print "Moving: '{}' -> '{}'".format(filepath, file_to)
        shutil.move(filepath, file_to)
      else:
    sys.stderr.write('Error: {} exists already.\n'.format(file_to))
  next_depth = depth - 1
  for subdirectory in (d for d in contained_filenames if os.path.isdir(d)):
    if is_directory_to_be_flattened(subdirectory):
      flatten(subdirectory, next_depth, to_directory, True)

def flatten_directory(to_flatten, depth):
  to_directory = os.path.join(to_flatten, FLATTED_DIRECTORY_NAME)
  if not os.path.isdir(to_flatten):
    sys.stderr.write(
      'The argument {} does not seem to be a directory.\n'.format(
      to_flatten))
    return
  flatten(to_flatten, depth, to_directory, False)

def main():
  if len(sys.argv) == 2:
    flatten_directory(sys.argv[1], DEFAULT_DEPTH)
  else:
    print 'usage: {} /path/to/directory/to/flatten'.format(sys.argv[0])

if __name__ == '__main__':
  main()

虽然从代码中可以明显看出,但意图是:

  • 从指定目录开始
  • 下降到一定深度
  • 考虑子目录并在其中移动所有文件,当且仅当:
    • 该目录包含一个具有给定文件名的“sentinel文件”
    • sentinel文件实际上是一个sentinel文件,而不仅仅是重命名为同名的文件
  • 整理搜索开始的目录下的__flattened__目录中的文件

1 个答案:

答案 0 :(得分:7)

Python中的大多数文件处理函数都在模块“os”中 - 您将在其中找到 os.rename(用于重命名或移动directoruy条目),os.listdir - 它为您提供目录中的文件名列表,作为第一个arg,os.walk传递 - 以递归方式遍历目录结构os.path.walk ,为了做同样的事情,但使用回调,os.path.exists,os.path.isdir,os.mkdir,是其他可能很方便的。

对于“快速而肮脏”的翻译,您也可以使用“os.system”。它允许你执行shell命令,就像它在shell中输入一样,而os.popen - 允许访问所述进程的stdin和stdout。更细致的翻译,很难,需要使用另一个模块:“subprocess”,它可以完全控制作为子进程执行的shell命令(例如,如果你需要find,它将无法使用在Windows上)

其他感兴趣的模块是sys(sys.argv是传递给脚本的参数)和shutil(包括copy,rmtree等)

你的脚本做了一些错误检查,并且它是微不足道的,考虑到“os”中的上述功能名称和基本的Python来添加它们 - 但是Python中的简单“只做它”脚本可能只是:

import os, sys

dir_to_flatten = sys.argv[1]

for dirpath, dirnames, filenames in os.walk(dir_to_flatten):
    for filename in filenames:
        try:
            os.rename(os.path.join(dirpath, filename), os.path.join(dir_to_flatten, filename))
        except OSError:
            print ("Could not move %s " % os.path.join(dirpath, filename))