Question

我的脚本部分基于此处的脚本：Upload files with FTP using PowerShell

这一切对于小文件都可以正常工作，但我正在尝试使用它来创建我们用于将访问mdb文件导出到只有ftp更强大的客户端的过程。

我的第一个测试涉及一个10MB的文件，我在Get-Content阶段遇到了System.OutOfMemoryException

在获取尝试期间，PowerShell ISE的运行速度接近2GIG。

这是一个完整的脚本示例（温柔。我对它很新）：

#####
# User variables to control the script
#####

# How many times connection will be re-tried
$connectionTries = 5
#time between tries in seconds
$connectionTryInterval = 300
#Where to log the output
$logFile = "D:\MyPath\ftplog.txt"
#maximum log file size in KB before it is archived
$logFileMaxSize = 500

#formatted date part for the specific file to transfer
#This is appended to the filename base. Leave as "" for none
$datePart = ""
#base part of the file name
$fileNameBase = "Myfile"
#file extension
$fileExtension = ".mdb"
#location of the source file (please include trailing backslash)
$sourceLocation = "D:\MyPath\"

#location and credentials of the target ftp server    
$userName = "iamafish"
$password = "ihavenofingers"
$ftpServer = "10.0.1.100"

######
# Main Script
#####

#If there is a log file and it is longer than the declared limit then archive it with  the current timestamp
if (test-path $logfile)
{
    if( $((get-item $logFile).Length/1kb) -gt $logFileMaxSize)
    {
        write-host $("archiving log to ftplog_" + (get-date -format yyyyMMddhhmmss) +     ".txt")
        rename-item $logFile $("ftplog_" + (get-date -format yyyyMMddhhmmss) + ".txt")
    }
}

#start new log entry
#Add-Content $logFile "___________________________________________________________"
#write-host $logEntry

#contruct source file and destination uri
$fileName = $fileNameBase + $datePart + $fileExtension
$sourceFile = $sourceLocation + $fileName
$sourceuri = "ftp://" + $ftpServer + "/" + $fileName


# Create a FTPWebRequest object to handle the connection to the ftp server
$ftprequest = [System.Net.FtpWebRequest]::create($sourceuri)

# set the request's network credentials for an authenticated connection
$ftprequest.Credentials = New-Object System.Net.NetworkCredential($username,$password)

$ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftprequest.UseBinary = $true
$ftprequest.KeepAlive = $false

$succeeded = $true
$errorMessage = ""

# read in the file to upload as a byte array
trap [exception]{
    $script:succeeded = $false
    $script:errorMessage = $_.Exception.Message
    Add-Content $logFile $((get-Date -format "yyyy-MM-dd hh:mm:ss") + "|1|" +    $_.Exception.Message)
    #write-host $logEntry
    #write-host $("TRAPPED: " + $_.Exception.GetType().FullName)
    #write-host $("TRAPPED: " + $_.Exception.Message)
    exit
}
#The -ea 1 forces the error to be trappable
$content = gc -en byte $sourceFile -ea 1


$try = 0

do{
    trap [System.Net.WebException]{
        $script:succeeded = $false
        $script:errorMessage = $_.Exception.Message
        Add-Content $logFile $((get-Date -format "yyyy-MM-dd hh:mm:ss") + "|1|" +    $_.Exception.Message)
        #write-host $logEntry
        #write-host $("TRAPPED: " + $_.Exception.GetType().FullName)
        $script:try++
        start-sleep -s $connectionTryInterval
        continue
        }
        $ftpresponse = $ftprequest.GetResponse()

} while(($try -le $connectionTries) -and (-not $succeeded))

if ($succeeded) { 

    Add-Content $logFile $((get-Date -format "yyyy-MM-dd hh:mm:ss") + "|0|" +    "Starting file transfer.")
    # get the request stream, and write the bytes into it
    $rs = $ftprequest.GetRequestStream()
    $rs.Write($content, 0, $content.Length)
    # be sure to clean up after ourselves
    $rs.Close()
    $rs.Dispose()
    $content.Close()
    $content.Dispose()
    Add-Content $logFile $((get-Date -format "yyyy-MM-dd hh:mm:ss") + "|0|" +    "Transfer complete.")
    #write-host $logEntry
}

我不能把代码放在注释中，所以，感谢keith的指针，我已经将文件访问位向下移动到底部，将其与其他类似的链接，如此...

trap [Exception]{
    $script:succeeded = $false
    $script:errorMessage = $_.Exception.Message
    Add-Content $logFile $((get-Date -format "yyyy-MM-dd hh:mm:ss") + "|1|Check File Connection|" + $_.Exception.Message)
    $sourceStream.Close()
    $sourceStream.Dispose()
    #write-host $((get-Date -format "yyyy-MM-dd hh:mm:ss") + "|1|Attempt to open file|" + $_.Exception.Message)
    #write-host $("TRAPPED: " + $_.Exception.GetType().FullName)
    exit
}
$sourceStream = New-Object IO.FileStream ($(New-Object System.IO.FileInfo $sourceFile),[IO.FileMode]::Open)
[byte[]]$readbuffer = New-Object byte[] 1024

# get the request stream, and write the bytes into it
$rs = $ftprequest.GetRequestStream()
do{
    $readlength = $sourceStream.Read($readbuffer,0,1024)
    $rs.Write($readbuffer,0,$readlength)
} while ($readlength -ne 0)

我只需要找出原因：使用“0”参数调用“GetResponse”的异常：“无法访问已处置的对象。 我每隔一次运行它。这是在ISE中运行它的怪癖，还是我在初始声明或最终处置方面做了一些严重的错误？

我会在完成后发布完整的最终脚本，因为我认为它会使用错误捕获和记录来制作一个很好的强大的ftp导出示例。

好的，这是完整的脚本。 Dispose已被删除，但有或没有它在5分钟内运行脚本将给我一个消息，我不能使用处置的opject或告诉我getResponse（）已产生错误（226）文件传输（在ISE中运行）。虽然这在正常操作期间不会出现问题，但我想正确记录FTP会话并清理脚本末尾的资源，并确保我正确地根据需要声明它们。

#####
# User variables to control the script
#####

# How many times connection will be re-tried
$connectionTries = 5
#time between tries in seconds
$connectionTryInterval = 1
#Where to log the output
$logFile = "D:\MyPath\ftplog.txt"
#maximum log file size in KB before it is archived
$logFileMaxSize = 500
#log to file or console - #true=log to file, #false = log to console
$logToFile=$false

#formatted date part for the specific file to transfer
#This is appended to the filename base. Leave as "" for none
$datePart = ""
#base part of the file name
$fileNameBase = "MyFile"
#file extension
$fileExtension = ".mdb"
#location of the source file (please include trailing backslash)
$sourceLocation = "D:\MyPath\"

#location and credentials of the target ftp server
$userName = "iamafish"
$password = "ihavenofingers"
$ftpServer = "10.0.1.100"

######
# Main Script
#####

function logEntry($entryType, $section, $message)
{
    #just to make a one point switch for logging to console for testing
    # $entryType: 0 = success, 1 = Error
    # $section: The section of the script the log entry was generated from
    # $message: the log message

    #This is pipe separated to fit in with my standard MSSQL linked flat file schema for easy querying
    $logString = "$(get-Date -format "yyyy-MM-dd hh:mm:ss")|$entryType|$section|$message"

    if($script:logtoFile)
    {
        Add-Content $logFile $logString
    }
    else
    {
        write-host $logString
    }
}

#If there is a log file and it is longer than the declared limit then archive it with the current timestamp
if (test-path $logfile)
{
    if( $((get-item $logFile).Length/1kb) -gt $logFileMaxSize)
    {
        write-host $("archiving log to ftplog_" + (get-date -format yyyyMMddhhmmss) + ".txt")
        rename-item $logFile $("ftplog_" + (get-date -format yyyyMMddhhmmss) + ".txt")
        New-Item $logFile -type file
    }
}
else
{
    New-Item $logFile -type file
}


#contruct source file and destination uri
$fileName = $fileNameBase + $datePart + $fileExtension
$sourceFile = $sourceLocation + $fileName
$destination = "ftp://" + $ftpServer + "/" + $fileName


#Check if the source file exists
if ((test-path $sourceFile) -eq $false)
{
    logEntry 1 "Check Source File" $("File not found: " + $sourceFile)
    Exit
}


# Create a FTPWebRequest object to handle the connection to the ftp server
$ftpRequest = [System.Net.FtpWebRequest]::create($destination)

# set the request's network credentials for an authenticated connection
$ftpRequest.Credentials = New-Object System.Net.NetworkCredential($username,$password)
$ftpRequest.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftpRequest.UseBinary = $true
$ftpRequest.KeepAlive = $false

$succeeded = $true
$try = 1

do{
    trap [Exception]{
        $script:succeeded = $false
        logEntry 1 "Check FTP Connection" $_.Exception.Message
        $script:try++
        start-sleep -s $connectionTryInterval
        continue
        }
        $ftpResponse = $ftpRequest.GetResponse()

} while(($try -le $connectionTries) -and (-not $succeeded))

if ($succeeded) {
    logEntry 0 "Connection to FTP" "Success"


    # Open a filestream to the source file
    trap [Exception]{
        logEntry 1 "Check File Connection" $_.Exception.Message
        $sourceStream.Close()
        $ftpResponse.Close()
        exit
    }
    $sourceStream = New-Object IO.FileStream ($(New-Object System.IO.FileInfo $sourceFile),[IO.FileMode]::Open)
    [byte[]]$readbuffer = New-Object byte[] 1024

    logEntry 0 "Starting file transfer" "Success"
    # get the request stream, and write the bytes into it
    $rs = $ftpRequest.GetRequestStream()
    do{
        $readlength = $sourceStream.Read($readbuffer,0,1024)
        $rs.Write($readbuffer,0,$readlength)
    } while ($readlength -ne 0)

    logEntry 0 "Transfer complete" "Success"
    # be sure to clean up after ourselves
    $rs.Close()
    #$rs.Dispose()
    $sourceStream.Close()
    #$sourceStream.Dispose()

}
$ftpResponse.Close()

尝试在最后捕获Transfer OK响应的示例：

logEntry 0 "Starting file transfer" "Success"
# get the request stream, and write the bytes into it
$rs = $ftpRequest.GetRequestStream()
do{
    $readlength = $sourceStream.Read($readbuffer,0,1024)
    $rs.Write($readbuffer,0,$readlength)
} while ($readlength -ne 0)
$rs.Close()
#start-sleep -s 2

trap [Exception]{
    $script:succeeded = $false
    logEntry 1 "Check FTP Connection" $_.Exception.Message
    continue
}
$ftpResponse = $ftpRequest.GetResponse()

Answer 1

我自己在使用RAM时遇到类似的问题，GB上传了一个3MB的文件，我发现更换了：

 $content = gc -en byte $sourceFile

使用：

 $content = [System.IO.File]::ReadAllBytes($sourceFile)

提供更好的表现。正如其他地方所提到的，分块对于真正的大文件来说是一个更好的解决方案，因为那时你并没有将整个文件同时保存在内存中，但上面的代码至少只消耗〜（文件大小））RAM的字节，这意味着它应该可以达到MB范围的~10s。

Answer 2

不是使用Get-Content将整个文件读入内存，而是尝试一次读取一个块并将其写入FTP请求流。我会使用一个较低级别的.NET文件流API来进行读取。不可否认，你不会认为10MB会造成内存问题。

此外，请确保在确定请求流并写入请求后获得响应。获取响应流是上传数据的内容。来自文档：

使用FtpWebRequest对象时   您必须将文件上传到服务器   将文件内容写入请求   通过调用获得的流   GetRequestStream方法或其方法   异步对应的，   BeginGetRequestStream和   EndGetRequestStream方法。你必须   写入流并关闭   在发送请求之前流。

请求被发送到服务器   调用GetResponse方法或其方法   异步对应的，   BeginGetResponse和EndGetResponse   方法。当请求的操作   完成，一个FtpWebResponse对象是   回。 FtpWebResponse对象   提供操作的状态   以及从中下载的任何数据   服务器

Powershell FTP发送大文件System.OutOfMemoryException

2 个答案: