删除顺序,相同,重复的文件

时间:2011-04-06 16:10:26

标签: language-agnostic scripting file duplicate-removal windows-scripting

我有一台运行Windows Server 2003 R2 Enterprise的服务器,目录中的每个目录都有50,000到250,000个1KB文本文件。文件名是顺序的(例如,MLLP000001.rcv,MLLP000002.rcv等),相同的文件将是顺序的。一旦后续文件不同,我可以预期我不会收到另一个相同的文件。

我需要一个可以执行以下操作的脚本,但我不知道从哪里开始。

for each file in the target directory index 'i'
{
  for each file in the target directory index 'j' = i+1
  {
    compare the hash values of files i and j

    if the hashes are identical
      delete file j
    if the hashes differ
      set i = j // to skip past the files that are now deleted
      break
  }
}

我试过DOS批处理脚本,但这真的很麻烦,我无法突破内循环,并且它会自行跳过,因为外部循环在目录中有一个文件列表,但该列表不断变化。据我所知,VBScript没有哈希函数。

2 个答案:

答案 0 :(得分:1)

由于文件大小只有1KB,为什么不进行逐位比较并避免哈希?

答案 1 :(得分:0)

听起来你可以这样做:

Set Files to an array of files in a given directory.
Set PreviousHash to hash of the first file in the Files.

For each CurrentFile file after the first in Files,
    Set CurrentHash to hash of the CurrentFile.
    If CurrentHash is equal to PreviousHash, then delete CurrentFile.
    Else, set PreviousHash to CurrentHash.