将脚本输出从一个巨大的CSV打破到较小的CSV

时间:2015-08-05 21:20:20

标签: powershell csv output

目前有一个脚本将运行服务器上的所有用户文件,并根据上次访问时间输出它们。客户需要报告去年未触及的文件。

问题是,报告太大了。我根据我的测试样本集估算了超过200万行CSV。因此,我喜欢将此脚本转换为输出许多较小的报告,而不是一个 HUGE 文件,按用户细分。

扫描root用户目录,使用该用户名吐出CSV,然后迭代到下一个用户,然后重复。

$cutOffDate = (Get-Date).addYears(-1)
$arr = @()
$exclusions = @(".lnk",".url",".ini",".odc",".ctx",".upd",".ica")

gci "D:\USER_FILES\company\USERS\Lname, Fname" -Recurse | ? {
  $_.PSIsContainer -eq $False -and
  $_.LastAccessTime -le $cutOffDate -and
  $exclusions -notcontains $_.Extension -and
  $_.length -gt "0" -and
  $_.Directory -notmatch ".*USERS\\.*\\Personal\\sysdata\\cookies"
} | % {
  $obj = New-Object PSObject
  $obj | Add-Member NoteProperty Directory $_.DirectoryName
  $obj | Add-Member NoteProperty Name $_.Name
  $obj | Add-Member NoteProperty MB ("{0:N3}" -f ($_.Length/1MB))
  $obj | Add-Member NoteProperty created $_.creationtime
  $obj | Add-Member NoteProperty LastAccessed $_.LastAccessTime
  $obj | Add-Member NoteProperty LastMofified $_.LastWriteTime
  $obj | Add-Member NoteProperty Extension $_.Extension
  $arr += $obj
}

$arr | Export-CSV -notypeinformation "C:\Output.csv"

以下是编辑 - 输出到管道而不是数组

$cutOffDate = (Get-Date).addYears(-1)
$exclusions = @(".lnk",".url",".ini",".odc",".ctx",".upd",".ica")

Get-ChildItem 'D:\USER_FILES\company\USERS' | ? { $_.PSIsContainer } | % {
  $name = $_.Name
  Get-ChildItem $_.FullName -Recurse |
    ? {
      $exclusions -notcontains $_.Extension -and
      $_.PSIsContainer -eq $false -and
      $_.LastAccessTime -le $cutOffDate -and
      $_.length -gt "0" -and
      $_.Directory -notmatch '.*USERS\\.*\\Personal\\sysdata\\cookies'
    } |
    select DirectoryName, Name, @{n='MB';e={"{0:N3}" -f ($_.Length/1MB)}},
           CreationTime, LastAccessTime, LastWriteTime, Extension |
    Export-Csv "D:\User-Files-Output\$name.csv" -NoType
}

Here is a screenshot of it writing the CSV as soon as it hits the conditions statement

注意:忽略初始路径,我现在正在diff环境中测试,所以C盘或D,根路径仍然是USERS。

2 个答案:

答案 0 :(得分:1)

枚举用户文件夹(假设您的用户文件夹是Lastname, Firstname内名为D:\USER_FILES\company\USERS的文件夹),然后为每个用户递归。另外,不要在循环中附加到数组。只需选择您的属性并将输出传递到Export-Csv即可。如果您有PowerShell v3或更新版本,则可以使用参数-File-DirectoryGet-ChildItem的输出分别限制为文件或文件夹。

Get-ChildItem 'D:\USER_FILES\company\USERS' -Directory | % {
  $name = $_.Name
  Get-ChildItem $_.FullName -Recurse -File |
    ? {
      $_.LastAccessTime -le $cutOffDate -and
      $exclusions -notcontains $_.Extension -and
      $_.length -gt "0" -and
      $_.Directory -notmatch '.*USERS\\.*\\Personal\\sysdata\\cookies'
    } |
    select DirectoryName, Name, @{n='MB';e={"{0:N3}" -f ($_.Length/1MB)}},
           CreationTime, LastAccessTime, LastWriteTime, Extension |
    Export-Csv "C:\$name.csv" -NoType
}

如果您仅限于PowerShell v2,请将上述内容更改为:

Get-ChildItem 'D:\USER_FILES\company\USERS' | ? { $_.PSIsContainer } | % {
  $name = $_.Name
  Get-ChildItem $_.FullName -Recurse |
    ? {
      -not $_.PSIsContainer -and
      $_.LastAccessTime -le $cutOffDate -and
      $exclusions -notcontains $_.Extension -and
      $_.length -gt "0" -and
      $_.Directory -notmatch '.*USERS\\.*\\Personal\\sysdata\\cookies'
    } |
    select DirectoryName, Name, @{n='MB';e={"{0:N3}" -f ($_.Length/1MB)}},
           CreationTime, LastAccessTime, LastWriteTime, Extension |
    Export-Csv "C:\$name.csv" -NoType
}

答案 1 :(得分:1)

移动最后一行的逻辑:

 $arr | Export-CSV -notypeinformation "C:\Output.csv"

进入最终ForEach-Object脚本块:

$obj | Export-Csv -NoTypeInformation -Path ("C:\{0}.csv" -f $user)

您需要从每个项目的路径中获取用户名称,例如

$_.FullName -match 'D:\\USER_FILES\\company\\USERS\\([^\\]+)\\' | Out-Null
$user = $Matches[1]

因此,您的最终ForEach-Object脚本块如下所示:

$_.FullName -match 'D:\\USER_FILES\\company\\USERS\\([^\\]+)\\' | Out-Null
$user = $Matches[1]    
New-Object PSObject |
    Add-Member NoteProperty Directory $_.DirectoryName -PassThru |
    Add-Member NoteProperty Name $_.Name -PassThru |
    Add-Member NoteProperty MB ("{0:N3}" -f ($_.Length/1MB)) -PassThru |
    Add-Member NoteProperty created $_.creationtime -PassThru |
    Add-Member NoteProperty LastAccessed $_.LastAccessTime -PassThru |
    Add-Member NoteProperty LastMofified $_.LastWriteTime -PassThru |
    Add-Member NoteProperty Extension $_.Extension -PassThru |
    Export-Csv -NoTypeInformation -Path ('C:\{0}.csv' -f $user)