Question

我正在尝试将不同主机上各种目录中的大量文件复制到一个目录中。由于结果非常耗时，而且我必须多次运行此程序，因此我更改了代码以利用多线程的帮助，每个线程通过一个进程执行XCOPY。

然而，我发现当我进行多线程复制时，一些文件（少于100，每次都不一样）不会被复制。我还没想到是因为当XCOPYing到同一个目录时多个进程相互干预，或者更多的是XCOPY问题（不确定它在同一目的地上支持多进程调用的程度）。

List<Thread> threadList = new List<Thread>;

foreach(FileEntry fileEntry in fileEntries)
{
    Thread thread = new Thread(() => {
        //full path of file, like \\host\directory\directoryB\fileA.txt
        string filePath = fileEntry._filePath;

        //name of file, like fileA.txt
        string file = fileEntry._file;

        //dumpDirectory is where I want to copy all the files to
        string destPath = Path.Combine(dumpDirectory, file);

        //each file here is either a directory or a real file, bad naming convention I know...(open to suggestions if any)
        string fileType = File.Exists(filePath) ? "f" : "d";

        using (Process process = new Process())
        {
            process.StartInfo.CreateNoWindow = true;
            process.StartInfo.UseShellExecute = false;
            process.StartInfo.FileName = "cmd.exe";

            process.StartInfo.Arguments = string.Format(@"/C echo {0} | XCOPY /E /C /H /R /K /O /Y {1} {2}", fileType, filePath, destPath);

            Console.WriteLine("copying {2} [{0}] to destination [{1}]", filePath, destPath, string.Compare(fileType, "f") == 0 ? "file" : "directory");
            process.Start();
            process.WaitForExit();

            if (process.ExitCode != 0)
            {
                Console.WriteLine("encountered problems when copying file [{0}]", filePath);
            }
        }
    });

    thread.Start();
    threadList.Add(thread);
}

foreach(Thread thread in threadList)
{
    thread.Join();
}

--------------------------- Bug修复如下---------------- ---------------

正如下面的回复所示（感谢各位快速回复，保存了我的周末^ _ ^），我重定向了流程输出，发现问题是“共享违规\ n无法创建目录”。将多个文件复制到同一目录时，xcopy出现问题（各种xcopy进程似乎在它们都检测到系统上不存在该目录时同时创建目录）。

将手动线程管理更改为Parallel.Foreach解决了问题并使代码看起来更好（尽管我还没弄清楚为什么它不会产生同样的问题）

或者，更糟糕的解决方法是使用EventWaitHandle包装process.Start（）。我使用了基于目标目录的不同EventWaitHandle实例，而不是使用一个句柄，因为另一种方式将无法使用多进程

//name the wait handle based on destination value
EventWaitHandle destWaitHandle = new EventWaitHandle(true, EventResetMode.AutoReset, string.Format("waitHandle{0}", destPath.GetHashCode()));

//only trigger the wait handle lock if the file being copied from (and to) is a directory. Based on the error log info XCopy seemed to never have issues with copying individual files
//and the program worked fine if I only put the lock at directory level
if (string.Compare(fileType, "d") == 0)
{
    destWaitHandle.WaitOne();
}

process.Start();

//this line was added to write error messages into respective files during the process' execution
process.BeginErrorReadLine();
process.WaitForExit();

if (string.Compare(fileType, "d") == 0)
{
    destWaitHandle.Set();
}

Answer 1

首先，我建议您切换到面向任务的代码，因为在这样的数字中创建线程可能会因为上下文切换而导致性能降级。

您可以使用并行扩展轻松完成此操作，如下所示：

Parallel.ForEach(fileEntries, fileEntry => { // YOUR code here };

其次，您可以通过检查XCOPY对象的StandardError属性来调查Process进程中出现的错误，如@x ...建议，something like this （你还必须重定向它才能阅读它）：

process.StartInfo.RedirectStandardError = true;
// ...
if (process.ExitCode != 0)
{
    Console.WriteLine("encountered problems when copying file [{0}]", filePath);
    Console.WriteLine(process.StandardError.ReadToEnd());
}

多个线程生成的进程将文件复制到同一目录时缺少文件条目C＃

1 个答案: