Question

我需要使用RCurl将超过65个文件上传到我的托管ftp服务器，但是在上传了几个文件后我收到错误：

 < 421 Too many connections (8) from this IP
* We got a 421 - timeout!
* Closing connection 291

根据我在日志中看到的Connection #283 to host ftp.myserver.com left intact，所以在连接失效15分钟< 220 You will be disconnected after 15 minutes of inactivity.之后连接失效之前不需要进一步授权。

至少这是我的理解，因为出于某种原因，它会在第二个文件之后立即重新发送USER和PASSWORD：

> USER admin@myserver.com
< 331 User admin@myserver.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /

我知道有多个授权请求（每个文件一个），所以我充斥着服务器。

是否有可能首先打开与FTP服务器的连接然后上传所有文件？比如FileZilla？

请找到我使用的R代码：

for(file in list.files(localPath)) {

  ftpUpload(paste0(localPath,file), userpwd = userpwd, to=paste0("ftp://", ftpPath, file), verbose=TRUE)

}

另一个SO成员在这里遇到了同样的问题，但看起来我已经使用了类似于提供给他的回复的东西而且它不起作用。 Using R to upload many files

谢谢

编辑（添加日志）：

> for(file in list.files(localPath)) {
+   
+   print(paste0("ftp://", ftpPath, file))
+   ftpUpload(paste0(localPath,file), userpwd = userpwd, to=paste0("ftp://", ftpPath, file), verbose=TRUE)
+ 
+ }
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation-1.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#310)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 2 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER cadmin@mywebsite.com
< 331 User cadmin@mywebsite.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||44516|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 44516
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#310)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation-1.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.180 seconds (measured here), 1.15 Kbytes per second
* Connection #310 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation-2.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#311)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 3 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER cadmin@mywebsite.com
< 331 User cadmin@mywebsite.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||45077|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 45077
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#311)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation-2.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.207 seconds (measured here), 1.00 Kbytes per second
* Connection #311 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation-3.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#312)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 4 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER cadmin@mywebsite.com
< 331 User cadmin@mywebsite.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||40390|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 40390
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#312)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation-3.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.189 seconds (measured here), 1.12 Kbytes per second
* Connection #312 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation-4.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#313)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 5 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER cadmin@mywebsite.com
< 331 User cadmin@mywebsite.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||46329|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 46329
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#313)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation-4.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.187 seconds (measured here), 1.11 Kbytes per second
* Connection #313 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation_protected-1.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#314)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 6 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER cadmin@mywebsite.com
< 331 User cadmin@mywebsite.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||46556|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 46556
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#314)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation_protected-1.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.186 seconds (measured here), 2.75 Kbytes per second
* Connection #314 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation_protected-2.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#315)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 7 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER cadmin@mywebsite.com
< 331 User cadmin@mywebsite.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||43730|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 43730
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#315)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation_protected-2.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.190 seconds (measured here), 2.69 Kbytes per second
* Connection #315 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation_protected-3.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#316)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 8 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER cadmin@mywebsite.com
< 331 User cadmin@mywebsite.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||43399|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 43399
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#316)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation_protected-3.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
< 226-File successfully transferred
< 226 0.196 seconds (measured here), 2.61 Kbytes per second
* Connection #316 to host ftp.mywebsite.com left intact
[1] "ftp://ftp.mywebsite.com/data/csv/data_private_allocation_protected-4.csv"
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#317)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 9 of 1000 allowed.
< 220-Local time is now 08:50. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER cadmin@mywebsite.com
< 331 User cadmin@mywebsite.com OK. Password required
> PASS mypassword
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
> CWD portfolios
* ftp_perform ends with SECONDARY: 0
< 250 OK. Current directory is /portfolios
> CWD csv
< 250 OK. Current directory is /data/csv
> EPSV
* Connect data stream passively
< 229 Extended Passive mode OK (|||48536|)
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connecting to 11.22.33.44 (11.22.33.44) port 48536
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#317)
> TYPE I
< 200 TYPE is now 8-bit binary
> STOR data_private_allocation_protected-4.csv
< 150 Accepted data connection
* Remembering we are in dir "data/csv/"
[1] "ftp://ftp.mywebsite.com/data/csv/data_coins_average-1.csv"
< 226-File successfully transferred
< 226 0.184 seconds (measured here), 2.77 Kbytes per second
* Connection #317 to host ftp.mywebsite.com left intact
* Hostname was NOT found in DNS cache
*   Trying 11.22.33.44...
* Connected to ftp.mywebsite.com (11.22.33.44) port 21 (#318)
< 421 Too many connections (8) from this IP
* We got a 421 - timeout!
* Closing connection 318
Error in function (type, msg, asError = TRUE)  : 
  Uploading to a URL without a file name!

Answer 1

这是一种解决方法，而不是我的问题的解决方案，但它就像一个魅力。我创建了一个包含ftp命令的文本文件，然后使用R ftp启动Ubuntu system()。

ftpcommands.txt：

open ftp.mywebsite.com
user admin@mywebsite.com mypassword
lcd ~/R/data/csv
cd data/csv
prompt
mput *.csv
bye

然后是R命令：

system("ftp -n < ~/R/data/ftpcommands.txt")

Answer 2

您似乎正在为每个文件打开一个新连接，而不是如何使用文件传输协议。一些不同的解决方案：

在 R 代码中打开较少的连接。
限制 curl 使用的最大并发连接数。
在 FTP服务器的配置中增加允许的并发连接限制。 （只有在您确实需要时才这样做，正如我所提到的，不要为每个文件打开连接。）

检查this detailed answer以更好地了解问题。不要理解我错了，同时拥有一个多于一个连接的性能是好的，但对于每个文件都没有，这绝对是太多了。大多数FTP客户端使用2。

如何在R中上传多个文件而不会收到421太多的连接错误？

2 个答案: