Question

是否存在内置的cmdlet或其中的某些组合，当下载每个块时，该cmdlet允许我开始解压缩文件流吗？我有一个PowerShell脚本，需要下载一个大文件（10 GB），并且必须等到它完成后才能开始扩展...

$wc = New-Object net.webclient
$wc.Downloadfile($appDataSnapshotUri, "%DataSnapshotFileName%.zip") # this can take some time

Expand-Archive -Path "%DataSnapshotFileName%.zip" -DestinationPath Run # so can this

Answer 1

好吧，事实证明zip文件不需要完全下载即可解压缩，您可以压缩/解压缩流。 .Net中有一些内置功能可用于流压缩，但不适用于zip存档。您可以为此使用SharpZipLib库：

从https://www.nuget.org/packages/SharpZipLib/下载.nupckg 将文件提取到任何文件夹。您将需要lib / net45中的ICSharpCode.SharpZipLib.dll

下面是他们的示例的简化翻译： https://github.com/icsharpcode/SharpZipLib/wiki/Zip-Samples#unpack-a-zip-using-zipinputstream-eg-for-unseekable-input-streams

Add-Type -Path ".\ICSharpCode.SharpZipLib.dll"

$outFolder = ".\unzip"

$wc = [System.Net.WebClient]::new()

$zipStream = $wc.OpenRead("http://gitlab/test/test1/raw/master/sample.zip")

$zipInputStream = [ICSharpCode.SharpZipLib.Zip.ZipInputStream]::New($zipStream)

$zipEntry = $zipInputStream.GetNextEntry()

$fileName = $zipEntry.Name

$buffer = New-Object byte[] 4096

$sw = [System.IO.File]::Create("$outFolder\$fileName")

[ICSharpCode.SharpZipLib.Core.StreamUtils]::Copy($zipInputStream, $sw, $buffer)

$sw.Close()

它只会提取第一个条目，您可以添加一个while循环，此示例可以运行。

这是一个带有while循环的代码段，用于提取多个文件（在上面的示例中，将其放在$zipEntry = $zipInputStream.GetNextEntry()之后）：

While($zipEntry) {

$fileName = $zipEntry.Name

Write-Host $fileName

$buffer = New-Object byte[] 4096

$sw = [System.IO.File]::Create("$outFolder\$fileName")

[ICSharpCode.SharpZipLib.Core.StreamUtils]::Copy($zipInputStream, $sw, $buffer)

$sw.Close()

$zipEntry = $zipInputStream.GetNextEntry()

}

修改

这是我发现可以工作的地方...

Add-Type -Path ".\ICSharpCode.SharpZipLib.dll"

$outFolder = "unzip"

$wc = [System.Net.WebClient]::new()

$zipStream = $wc.OpenRead("https://github.com/Esri/file-geodatabase-api/raw/master/FileGDB_API_1.5/FileGDB_API_1_5_VS2015.zip")

$zipInputStream = [ICSharpCode.SharpZipLib.Zip.ZipInputStream]::New($zipStream)

$zipEntry = $zipInputStream.GetNextEntry()

while($zipEntry) {

if (-Not($zipEntry.IsDirectory)) { 
  $fileName = $zipEntry.Name

  $buffer = New-Object byte[] 4096

  $filePath = "$pwd\$outFolder\$fileName"
  $parentPath = "$filePath\.."
  Write-Host $parentPath

  if (-Not (Test-Path $parentPath)) {
      New-Item -ItemType Directory $parentPath
  }

  $sw = [System.IO.File]::Create("$pwd\$outFolder\$fileName")

  [ICSharpCode.SharpZipLib.Core.StreamUtils]::Copy($zipInputStream, $sw, $buffer)
  $sw.Close()

}

$zipEntry = $zipInputStream.GetNextEntry()

}

Answer 2

要扩展Mike Twc的答案，请使用脚本在有或没有流的情况下执行此操作，并比较所需时间：

$url = "yoururlhere"

function UnzipStream () {
    Write-Host "unzipping via stream"

    $stopwatch1 =  [system.diagnostics.stopwatch]::StartNew()


    Add-Type -Path ".\ICSharpCode.SharpZipLib.dll"

    $outFolder = "unzip-stream"

    $wc = [System.Net.WebClient]::new()

    $zipStream = $wc.OpenRead($url)

    $zipInputStream = [ICSharpCode.SharpZipLib.Zip.ZipInputStream]::New($zipStream)

    $zipEntry = $zipInputStream.GetNextEntry()

    while($zipEntry) {

    if (-Not($zipEntry.IsDirectory)) { 
    $fileName = $zipEntry.Name

    $buffer = New-Object byte[] 4096

    $filePath = "$pwd\$outFolder\$fileName"
    $parentPath = "$filePath\.."
    Write-Host $parentPath

    if (-Not (Test-Path $parentPath)) {
        New-Item -ItemType Directory $parentPath
    }

    $sw = [System.IO.File]::Create("$pwd\$outFolder\$fileName")

    [ICSharpCode.SharpZipLib.Core.StreamUtils]::Copy($zipInputStream, $sw, $buffer)

    $sw.Close()
    }

    $zipEntry = $zipInputStream.GetNextEntry()

    }

    $stopwatch1.Stop()

    Write-Host "extraction took $($stopWatch1.ElapsedMilliseconds) millis with stream"
}

function UnzipWithoutStream() {

    Write-Host "Extracting without stream"

    $stopwatch2 = [system.diagnostics.stopwatch]::StartNew()
    $outFolder2 = "unzip-normal"

    $wc2 = New-Object System.Net.WebClient
    $wc2.DownloadFile($url, "$pwd\download.zip")

    $of2 = New-Item -ItemType Directory $outFolder2

    Expand-Archive -Path "download.zip" -DestinationPath $of2.FullName

    $stopwatch2.Stop()

    Write-Host "extraction took $($stopWatch2.ElapsedMilliseconds) millis without stream"
}

UnzipStream
UnzipWithoutStream

Powershell解压缩流

2 个答案: