我正在尝试将大文件（1GB）上传到SFTP，但我不断得到一些变体：
File "/venv/lib/python2.7/site-packages/paramiko/file.py", line 339, in write
self._write_all(data)
File "/venv/lib/python2.7/site-packages/paramiko/file.py", line 456, in _write_all
count = self._write(data)
File "/venv/lib/python2.7/site-packages/paramiko/sftp_file.py", line 180, in _write
t, msg = self.sftp._read_response(req)
File "/venv/lib/python2.7/site-packages/paramiko/sftp_client.py", line 762, in _read_response
raise SSHException('Server connection dropped: %s' % str(e))
SSHException: Server connection dropped:
我注意到如果我将MAX_REQUEST_SIZE（在sftp_file.py中）更新为1024而不是32768，它就可以了。这是否意味着我唯一的选择是使用MAX_REQUEST_SIZE = 1024复制/粘贴自定义版本的sftp_file.py？有没有其他人有建议不会减慢上传速度？
更新：我最近几次尝试更新OperationalError: SSL SYSCALL error: EOF detected时，最终导致MAX_REQUEST_SIZE错误。作为参考，这正是我目前正在做的事情：
transport = paramiko.Transport((hostname, port))
transport.connect(username, password)
sftp = paramiko.SFTPClient.from_transport(transport)
f = sftp.open(ftp_path, 'wb')
f.write(file_obj.read())
f.close()
sftp.close()
transport.close()
答案 0 :(得分：0)
IMO，你很难减少大块头。 SSH的RFC4253声明：
所有实现必须能够处理带有的数据包 未压缩的有效载荷长度为32768字节或更少以及总包 大小为35000字节或更少（包括＆＃39; packet_length＆＃39;， ＆＃39; padding_length＆＃39;，＆＃39;有效负载＆＃39;，＆＃39;随机填充＆＃39;和＆＃39; mac＆＃39;）。
无论如何，你可以做的不是使用sftp_file.py的自定义版本，而是：
sftpclient = sshclient.open_sftp()
with sftpclient.file(remote_file_path, mode='w') as rem_file:
rem_file.MAX_REQUEST_SIZE = 1024
rem_file.write(data)
和MAX_REQUEST_SIZE一起玩（即16384,24576，......）
答案 1 :(得分：0)
我在使用 get 之前添加了 open/close 并且出于某种我不知道的原因，它适用于大文件。
file = self._connection.open(remote_path, 'rb')
file.close()
self._connection.get(remote_path, local_path,callback=None)
答案 2 :(得分：0)
我收到错误-“ERROR_SFTP_PULL, AUTHENTICATION ERROR 服务器连接断开”： - 正常无密码 sftp (get) 正在从我的本地服务器到远程服务器，但是 SFTP PULL 通过 Python 脚本失败。 4天前运行正常。 -下面的代码部分看起来很糟糕：
1. authReturnCode = "<class 'paramiko.ssh_exception.BadAuthenticationType'>", not sure please help
2. **errorKey = "INFO_SFTP_PULL, "+ eachFiles + " successfully copyied from SFX for " + custName**
Failure log:
===========
2021-04-26 15:00:02,459 INFO::ipndProxyApp.py:787: Processing data for: eDCP
2021-04-26 15:00:02,851 INFO::transport.py:1746: Connected (version 2.0, client SSHD)
2021-04-26 15:00:03,860 INFO::transport.py:1746: Authentication (publickey) successful!
2021-04-26 15:00:03,922 INFO::sftp.py:158: [chan 0] Opened sftp connection (server version 3)
2021-04-26 15:15:15,376 INFO::sftp.py:158: [chan 0] sftp session closed.
2021-04-26 15:15:15,491 ERROR::ipndProxyApp.py:119: ERROR_SFTP_PULL, AUTHENTICATION ERROR Server connection dropped:
Normal log is:
=============
2021-04-22 15:00:02,138 INFO::ipndProxyApp.py:782: Processing data for: eDCP
2021-04-22 15:00:02,256 INFO::transport.py:1746: Connected (version 2.0, client SSHD)
2021-04-22 15:00:02,563 INFO::transport.py:1746: Authentication (publickey) successful!
2021-04-22 15:00:02,586 INFO::sftp.py:158: [chan 0] Opened sftp connection (server version 3)
2021-04-22 15:00:09,999 INFO::sftp.py:158: [chan 0] sftp session closed.
2021-04-22 15:00:10,003 INFO::ipndProxyApp.py:122: INFO_SFTP_PULL, FILE_2021-04-21-18-00-35.txt successfully copied from SFX for eDCP
Python Code:
===========
import os, sys, logging, shutil
import pysftp, mysql.connector
import subprocess, csv, argparse
from datetime import datetime, timedelta
import time
# Importing Configuration settings or other supporting scripts
import settings
import SQLqueries
import fieldLengthVerification
# Static Configuration settings, logging settings, global variables
logging.basicConfig(filename='/var/log/ipnd.log', format='%(asctime)s %(levelname)s::%(filename)s:%(lineno)d: %(message)s', dateformat='%Y-%m-%d %I:%M:%S',level=logging.INFO)
currentDateTime = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
currentDate = datetime.now().strftime("%Y-%m-%d")
yesterdays_date = (datetime.now() - timedelta(days=1)).strftime("%Y-%m-%d")
twoDaysBack = (datetime.now() - timedelta(days=2)).strftime("%Y-%m-%d")
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
# Handling Authentication errors for the SFT authentications only
def errorHandling(errorFromProgram):
"""
Function to handle authentication error for sftp only, it accepts exception errors and return string keyword.
Parameters:
errorFromProgram (class) : Error class generated by Exceptions
Returns:
str: str(errorFromProgram) or "Authentication Success" or "AUTHENTICATION ERROR " + str(errorFromProgram)
"""
# This function to perform error handling and return string against the provided input
authReturnCode = "<class 'paramiko.ssh_exception.BadAuthenticationType'>"
# Handling RHEL version of sftp (<class 'paramiko.ssh_exception.AuthenticationException'>)
if str(errorFromProgram) == "Authentication failed.":
errorKeyWord = str(errorFromProgram)
return errorKeyWord
# Handling SUSE version of sftp (<class 'paramiko.ssh_exception.BadAuthenticationType'>)
elif str(type(errorFromProgram)) == authReturnCode:
errorExcetption,a = errorFromProgram
errorKeyWord = errorExcetption
return errorKeyWord
# Handling other situation then provious two
elif (str(errorFromProgram) != "Authentication failed.") or (str(type(errorFromProgram)) != authReturnCode):
errorKeyWord = "AUTHENTICATION ERROR " + str(errorFromProgram)
return errorKeyWord
# Handling other conditions if not handled in provious if and elif
else:
infoKeyWord = "Authentication Success"
return infoKeyWord
# Fetch eDCP Data files from SFX sftp server
def fetchFilesFromSftpLocation(custName, fileSrcHost, remoteFileLocation, fileSrcUser, fileSrcPassPhrase, dateToProcessFiles):
"""
Function to fetch files from remove servers and store in location directory, return nothing and accept following parameters
Parameters:
custName (str): Customer Name.
fileSrcHost (str): Hostname (FQDN) or IP address of the source server from where files would be fetched.
fileLocation (str): Remote file location.
fileSrcUser (str): User to connect remote server to fetch files.
fileSrcPassPhrase (str): Password to connect remote server to fetch files.
dateToProcessFiles (str): Date to fetch the files for
"""
# try to connect SFX server over sftp
try:
sftp = pysftp.Connection(fileSrcHost, username = fileSrcUser)
# change to remote directory
sftp.chdir(remoteFileLocation)
listOfFilesAtSFX = sftp.listdir()
for eachFiles in listOfFilesAtSFX:
if eachFiles.startswith("IPNDUP" + custName.upper() +"_" + dateToProcessFiles):
# Fetch files from SFTP
try:
sftp.get(eachFiles)
**errorKey = "INFO_SFTP_PULL, "+ eachFiles + " successfully copyied from SFX for " + custName**
except Exception as e:
errorKey = "ERROR_SFTP_PULL, "+ errorHandling(e)
#print(errorKey)
sftp.close()
# Capture Exception
except Exception as e:
errorKey = errorHandling(e)
errorKey = "ERROR_SFTP_PULL, "+ errorKey
# Checking conditions to generate ERROR or INFO
if len(errorKey) > 0:
listOfErrorKeys = errorKey.split('_')
if "ERROR" in listOfErrorKeys:
logging.error("%s" % errorKey)
sys.exit(1)
else:
logging.info("%s" % errorKey)
else:
logging.error("%s" % "No Error keywords found")