在Powershell中将Chrome Cookies用于Invoke-WebRequest

时间:2018-12-23 18:36:32

标签: powershell web cookies session-cookies scrape

是否可以将Powershell中现有的Chrome网络会话中已获取的Cookie用于SessionVariable?

我知道SessionVariable的字段是:

Headers               : {}
Cookies               : System.Net.CookieContainer
UseDefaultCredentials : False
Credentials           : 
Certificates          : 
UserAgent             : Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) 
                        WindowsPowerShell/5.1.17134.407
Proxy                 : 
MaximumRedirection    : -1

我最初可以为webrequest做些设置这些值的事情吗?当会话可能是多种/不同类型的请求时,是否需要特定的会话标题?

$ua = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"
    $Params = @{
        Headers = $headers 
        Cookies = $cookies 
        UseDefaultCredentials = "False" 
        Credentials = $creds
        UserAgent = $ua
        Proxy = ""
        MaximumRedirection = "-1"
    }

如果可以执行类似的操作,那么我对输入cookie的方式也有些困惑。我是从Getting Cookies using PowerShell找到的:

$webrequest = Invoke-WebRequest -Uri $url -SessionVariable websession 
$cookies = $websession.Cookies.GetCookies($url) 

# Here, you can output all of $cookies, or you can go through them one by one. 

foreach ($cookie in $cookies) { 
     # You can get cookie specifics, or just use $cookie 
     # This gets each cookie's name and value 
     Write-Host "$($cookie.name) = $($cookie.value)" 
}

我可以从任何给定站点以这种格式导出Cookie的JSON格式,但是如果需要的话,我也可以将其缩减为“ name” =“ value”。

[
  {
    "domain": "",
    "expirationDate": "",
    "hostOnly": "",
    "httpOnly": "",
    "name": "",
    "path": "",
    "sameSite": "",
    "secure": "",
    "session": "",
    "storeId": "",
    "value": "",
    "id": ""
  },
  {
    "domain": "",
    etc...
  }
]

我不知道如何在$ cookies中将每个部分指定为$ cookie,也不知道如何以不同的方式添加它们,因为它不是来自URL / WebRequest而是来自JSON。

感谢您的帮助!

1 个答案:

答案 0 :(得分:0)

这是使用 PSSQLite 作为 PowerShell 脚本的解决方案:

# One time setup
    # Download the repository 
        $RepositoryZipUrl = "https://api.github.com/repos/RamblingCookieMonster/PSSQLite/zipball/master"
        Invoke-RestMethod -Uri $RepositoryZipUrl -OutFile "PSSQLite.zip"
    # Unblock the zip
        Unblock-File "PSSQLite.zip"
    # Extract the PSSQLite folder to a module path (e.g. $env:USERPROFILE\Documents\WindowsPowerShell\Modules\)
        Expand-Archive -Path "PSSQLite.zip" -DestinationPath $(Resolve-Path "$env:USERPROFILE\Documents\*PowerShell\Modules" | Select-First 1) -Force -Confirm 
    #Simple alternative, if you have PowerShell 5, or the PowerShellGet module:
        Install-Module PSSQLite

# Import the PSSQlite module
    Import-Module PSSQLite    #Alternatively, Import-Module \\Path\To\PSSQLite
# Specify for which domain you want to retrieve cookies
    $domain = "mavaddat.ca"

# Create a SQLite connection to Cookies
    $cookiesSQL = New-SQLiteConnection -DataSource "$env:APPDATA\Opera Software\Opera Stable\Cookies"

# Investigate the db structure
    $cookiesSQL.GetSchema()

# Investigate tables' structures to formulate a SQL query
    $cookiesSQL.GetSchema("Tables")

# Based on the schema of table `cookies`, form the query
    $query = "SELECT `"_rowid_`",* FROM `"main`".`"cookies`" WHERE `"host_key`" LIKE '%$domain%' ESCAPE '\' LIMIT 0, 49999;"

# Read the cookies from the SQLite
    $cookies = Invoke-SqliteQuery -Query $query -DataSource $cookiesSQL.FileName

# Get Chromium cookie master key
    $cookiesKeyEncBaseSixtyFour = (Get-Content -Path "$env:APPDATA\Opera Software\Opera Stable\Local State" | ConvertFrom-Json).'os_crypt'.'encrypted_key'
    $cookiesKeyEnc = [System.Convert]::FromBase64String($cookiesKeyEncBaseSixtyFour) | Select-Object -Skip 5  # Magic number 5
    $cookiesKey = [System.Security.Cryptography.ProtectedData]::Unprotect($cookiesKeyEnc,$null,[System.Security.Cryptography.DataProtectionScope]::LocalMachine)

# Create a web session object for the IWR work
    $session = New-Object Microsoft.PowerShell.Commands.WebRequestSession

# Stuff the cookies into the session
    foreach($cookie in $cookies){
        $names = $cookie | Get-Member | Where-Object -FilterScript {($_.MemberType) -eq 'NoteProperty'} | Select-Object -Property Name 
        foreach($name in $names)
        {
            $path = if ([string]::IsNullOrEmpty($cookie.'path'))
            {
                '/'
            } 
            else 
            { 
                $cookie.'path'
            }
$cipherStream = [System.IO.MemoryStream]::new($cookie.encrypted_value)
$cipherReader = [System.IO.BinaryReader]::new($cipherStream)
$nonSecretPayload = $cipherReader.ReadBytes(3) # Magic number 3
$nonce = $cipherReader.ReadBytes([System.Security.Cryptography.AesGcm]::NonceByteSizes.MinSize)

            $session.Cookies.Add([System.Net.Cookie]::new(#)
        }
    }

# Now use the session to IWR
    $downloadToPath = "c:\somewhere\on\disk\file.zip"
    $remoteFileLocation = "http://somewhere/on/the/internet"
    Invoke-WebRequest $remoteFileLocation -WebSession $session -TimeoutSec 900 -OutFile $downloadToPath

我是如何实现上述目标的

Chromium-based browser 使用存储在二进制 SQLite 数据库文件中的 cookie。请参阅有关如何为 Chrome 查找此数据库的超级用户问题,"Is there a way to watch cookies and their values live?"

对我而言(在 Windows 10 上使用 Opera 74.0.3911.107),此文件名为 Cookies,位于 %appdata%\Opera Software\Opera Stable\Cookies。使用 PSSQLite,我可以看到数据库有两个具有以下架构的表:

Cookies

<头>
姓名 类型 架构
creation_utc 整数 "creation_utc" 整数非空
host_key 文本 "host_key" 文本非空
名称 文本 "name" TEXT NOT NULL
价值 文本 “值”文本非空
路径 文本 “路径”文本非空
expires_utc 整数 "expires_utc" 整数非空
is_secure 整数 "is_secure" 整数非空
is_httponly 整数 "is_httponly" 整数非空
last_access_utc 整数 "last_access_utc" 整数非空
has_expires 整数 "has_expires" INTEGER NOT NULL DEFAULT 1
is_persistent 整数 "is_persistent" INTEGER NOT NULL DEFAULT 1
优先级 整数 "priority" INTEGER NOT NULL DEFAULT 1
encrypted_value BLOB "encrypted_value" BLOB 默认值 ''
同站点 整数 "samesite" INTEGER NOT NULL DEFAULT -1
source_scheme 整数 "source_scheme" INTEGER NOT NULL DEFAULT 0
源端口 整数 "source_port" INTEGER NOT NULL DEFAULT -1
is_same_party 整数 "is_same_party" INTEGER NOT NULL DEFAULT 0

Meta

<头>
姓名 类型 架构
LONGVARCHAR "key" LONGVARCHAR NOT NULL UNIQUE
价值 LONGVARCHAR "value" LONGVARCHAR

参见 stackoverflow 问题 "How to read Brave Browser cookie database encrypted values in C# (.NET Core)?" 我也遵循了 this gist 的 PowerShell cookie 方法 by @lawrencegripper 作为模板。