使用大型csv文件加速powershell脚本

时间:2015-05-27 03:24:24

标签: powershell csv

我对powershell有点新意。现在只玩弄它几天并编写下面的脚本来帮助搜索csv文件中的多个条件。我在VB中写了类似的东西,处理csv文件需要2天。这个powershell脚本大约需要6个小时来处理6500台机器和9个策略。我想要做的是在Policies.csv中查看来自computers.csv的计算机和来自列表的策略,并报告计算机是否具有该策略。

Policy.csv表中有6个字段需要在最终报告中,并为策略的状态添加了一个附加字段。

Computers.csv在表中有2个字段,分别是计算机名称和所在的OU。

Packlist.txt只是正在查找的应用程序(策略)的列表。

编辑: csv文件的样本如下

Policy.csv  
Device,Device DN,Group,Group DN,Policy Domain,Policy  
Comp1,OU=Here,Domain_app,OU=Here,Ou=Apps,Server1,Policy1  
Comp1,OU=Here,Domain_app,OU=Here,Ou=Apps,Server1,Policy2  
Comp1,OU=Here,Domain_app,OU=Here,Ou=Apps,Server1,Policy3  
Comp2,OU=There,Domain_app,OU=Here,Ou=Apps,Server1,Policy1    
Comp2,OU=There,Domain_app,OU=Here,Ou=Apps,Server1,Policy2  
Comp2,OU=There,Domain_app,OU=Here,Ou=Apps,Server1,Policy3  

Computer.csv  
Device,Device DN  
Comp1,OU=Here  
Comp2,OU=There  
Comp3,OU=AnyWhere  

Packlist.txt  
Policy1  
Policy3  

Result.csv  
Device,Device DN,Group,Group DN,Policy Domain,Policy,Status
Comp1,OU=Here,Domain_app,OU=Here,Ou=Apps,Server1,Policy1,Entitled
Comp2,OU=There,Domain_app,OU=Here,Ou=Apps,Server1,Policy1,Entitled
Comp3,OU=AnyWhere,,,,Policy1,Notentitled
Comp1,OU=Here,Domain_app,OU=Here,Ou=Apps,Server1,Policy3,Entitled
Comp2,OU=There,Domain_app,OU=Here,Ou=Apps,Server1,Policy3,Entitled
Comp3,OU=AnyWhere,,,,Policy3,Notentitled

代码是:

$data=import-csv -path c:\packagestatus\policy.csv
$computers=import-csv -path c:\packagestatus\computers.csv
$policylist= (Get-content -path c:\packagestatus\packlist.txt)
$policycount = $Policylist.count
$computercount = $computers.count
$Policycounter = 1

foreach ($policy in $policylist) 
    {
    $Policy
    $host.ui.RawUI.WindowTitle = "Processing $policyCounter of $policycount"
    $Data_temp = $data|where-object{$_."Policy Instance" -eq $policy}
    $computercounter = 1
    foreach ($Computer in $computers)
        {
        $host.ui.RawUI.WindowTitle = "Processing Policy $policyCounter of $policycount and Computer $computercounter of $computercount"
        if ($data_temp|Where-Object{$_.Device -eq $computer.device})
            {
            $result = $data_temp|where-object{$_.Device -eq $computer.device}|Where-Object{$_."Policy Instance" -eq $policy}
            $result|Add-member -membertype Noteproperty -name Status -value Entitled
            $result|export-csv -path c:\packagestatus\result1.csv -NoTypeInformation -append
            }
        Else
            {
            $result1 = New-Object -TypeName PSObject
            $result1|add-member -membertype noteproperty -name "Device" -value $computer.device
            $result1|add-member -membertype noteproperty -name "Device DN" -value $computer."Device DN"
            $result1|add-member -membertype noteproperty -name "Group" -value $null
            $result1|add-member -membertype noteproperty -name "Group DN" -value $null
            $result1|add-member -membertype noteproperty -name "Policy Domain" -value $null
            $result1|add-member -membertype noteproperty -name "Policy Instance" -value $Policy
            $result1|add-member -membertype noteproperty -name "Status" -value NotEntitled
            $result1|export-csv -path c:\packagestatus\result1.csv -force   -NoTypeInformation -append
            }
        $computercounter++  
        }
    $policycounter++
    }
$host.ui.RawUI.WindowTitle = "Completed"

1 个答案:

答案 0 :(得分:0)

好的,我认为这对你来说应该更快......

我首先创建一个函数来为给定的计算机名称,DN和缺少的策略创建对象。然后我从文件中加载数据。接下来,我为$Policylist中的所有策略创建一个正则表达式字符串以匹配。我为计算机列表做同样的事情。然后,我只过滤$Data数组,仅查找策略列表中以及计算机列表中的条目。

希望这会限制我们正在处理的数据,而且我认为总的来说会更快。接下来,我按设备对其进行分组,对于每个分组,我查找任何缺少的策略,并针对该函数运行该列表,并且我添加“Status”属性并输出该条目的任何匹配策略。这全部收集在$Results数组中。

一旦我们处理了我们记录的所有计算机,我们就会查找列表中没有的计算机,并为所有策略创建一个NotEntitled对象,并为$Results创建所有这些对象。

最后,我们对$ Results进行排序和输出。我认为不对它进行排序会更快,但也可能更难阅读。这是代码,让我知道它是如何为你工作的:

Function NotEntitled{
[CmdletBinding()]
Param(
    [String]$Device,
    [String]$DeviceDN,
    [String[]]$Pack
)
Process{
    ForEach($Item in $Pack){
        [PSCustomObject]@{
            'Device' = $Device
            'Device DN' = $DeviceDN
            'Group' = $null
            'Group DN' = $null
            'Policy Domain' = $null
            'Policy' = $Item
            'Status' = 'NotEntitled'
        }
    }
}
}

$Data = import-csv -path c:\packagestatus\policy.csv
$Computers = import-csv -path c:\packagestatus\computers.csv
$Policylist = ,(Get-content -path c:\packagestatus\packlist.txt)

$PolicyReg = ($Policylist|%{[regex]::Escape($_)}) -join '|'
$ComputerReg = ($Computers.Device|%{[regex]::Escape($_)}) -join '|'

$FilteredData = $Data | Where{$_.Policy -match $PolicyReg -and $_.device -match $ComputerReg}

$Results = $FilteredData | Group Device | ForEach{
    $Device = $_.group
    $MissingPolicies = ,($Policylist | Where{$_ -notin $Device.Policy})
    If(![string]::IsNullOrEmpty($MissingPolicies)){NotEntitled $Device[0].Device $Device[0].'Device DN' $MissingPolicies}
    $Device | ForEach{Add-Member -InputObject $_ -NotePropertyName 'Status' -NotePropertyValue 'Entitled' -PassThru}
}

$CompList = $FilteredData | Select -ExpandProperty Device -Unique

$Results += $Computers | Where{$_.Device -notin $CompList} | ForEach{NotEntitled $_.Device $_.'Device DN' $Policylist}

$Results | Sort Device,Policy | Export-Csv c:\packagestatus\Result.csv -NoTypeInformation

我获取了您的示例数据,将Comp1 Policy3更改为Comp1 Policy4(这样我就可以拥有一台只有部分策略集的计算机),并运行它并获得这些结果输出:

"Device","Device DN","Group","Group DN","Policy Domain","Policy","Status"
"Comp1","OU=Here","Domain_app","OU=Here,Ou=Apps","Server1","Policy1","Entitled"
"Comp1","OU=Here",,,,"Policy3","NotEntitled"
"Comp2","OU=There","Domain_app","OU=Here,Ou=Apps","Server1","Policy1","Entitled"
"Comp2","OU=There","Domain_app","OU=Here,Ou=Apps","Server1","Policy3","Entitled"
"Comp3","OU=AnyWhere",,,,"Policy1","NotEntitled"
"Comp3","OU=AnyWhere",,,,"Policy3","NotEntitled"