Question

使用s3client PHP SDK同步目录时效果很好

$s3client->uploadDirectory('/local/directory', 'my-bucket');

我仍然需要s3cmd --delete-removed功能来删除存储桶中存在但不存在于源目录中的不匹配文件。

REF：http://blogs.aws.amazon.com/php/post/Tx2W9JAA7RXVOXA/Syncing-Data-with-Amazon-S3

Answer 1

列出现有文件及其md5sum值
获取所有s3对象
比较md5s，如果不同则转移

删除任何剩余的＆＃39; （ - 删除删除）文件

function syncS3($bucket, $prefix, $targetDir)
{
    if (!is_dir($targetDir)) {
        mkdir($targetDir, 0755, true);
    }

    $localFileM5Sums = getMd5ForFilesInDir($targetDir);
    $s3Objects       = $this->s3Client->listObjects(['Bucket' => $bucket, 'Prefix' => $prefix]);

    foreach ($s3Objects['Contents'] as $contents) {
        $s3Key = $contents['Key'];
        if (substr($s3Key, -1, 1) == '/') { // is a 'directory'
            continue; // skip
        }

        $saveAs = $targetDir . DIRECTORY_SEPARATOR . $s3Key;
        if (isset($localFileM5Sums[$saveAs])) { // if we have local copy
            $localMd5 = $localFileM5Sums[$saveAs];
            unset($localFileM5Sums[$saveAs]); // remove so we don't try to delete it later
            if ($localMd5 === trim($contents['ETag'], '"')) { // and md5sumMatches
                continue; // don't transfer
            }
        }

        // transfer it
        if (!is_dir(dirname($saveAs))) {
            mkdir(dirname($saveAs), 0755, true);
        }
        $this->s3Client->getObject([
            'Bucket' => $bucket,
            'Key' => $s3Key,
            'SaveAs' => $saveAs
        ]);
    }

    // remove any missing files
    foreach ($localFileM5Sums as $fileName => $md5sum) {
        unlink($fileName);
    }
}


function getMd5ForFilesInDir($targetDir)
{
    $iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($targetDir));
    $files    = [];
    foreach ($iterator as $file) {
        if (!$file->isDir())
            $files[$file->getPathname()] = md5_file($file->getPathname());
    }
    return $files;
}

PHP：与Amazon S3同步数据

1 个答案: