使用apache FTPClient从FTP服务器下载文件

时间:2014-09-26 20:30:53

标签: java ftp download apache-commons ftp-client

我正在使用apache的FTPClient从FTP服务器下载文件。我的方案是 - FTP服务器可能会丢失网络连接,并可能最多保持断开连接1天。当它连接回来时,文件的下载应该从它离开的地方开始。我使用以下代码连接到服务器,然后从服务器下载文件

public void retrieve(String server, int port, String username,
        String password, String remote, String local, int fileType,
        ProgressHandler progressHandler) throws Exception {     
    final FTPClient ftp = new FTPClient();
    Date date = new Date();
    long startTime_ms = date.getTime();
    if (progressHandler != null) {
        ftp.setCopyStreamListener(new FtpCopyStreamListener(progressHandler));
    }

    ftpConnect(server,ftp, port,startTime_ms);


    if(ftp.getReplyCode()==0 || !String.valueOf(ftp.getReplyCode()).startsWith("2")){
        cleanup(ftp, "Could not log into server: " + server, null);
        return;
    }

    boolean loggedIn = false;
    try {
        if (username == null || username.isEmpty()) {
            username = "anonymous";
            password = System.getProperty("user.name") + "@"
                    + InetAddress.getLocalHost().getHostName();
        }
        if (!ftp.login(username, password)) {
            ftp.logout();
            cleanup(ftp, "Could not log into server: " + server, null);
        }
        loggedIn = true;

        ftp.setFileType(fileType);
        ftp.enterLocalPassiveMode();
        OutputStream output = null;
        try {
            output = new FileOutputStream(local);
            LOGGER.info("About to download " + remote + " from " + server
                    + " on " + (port > 0 ? port : ftp.getDefaultPort())
                    + " to " + local);              
            ftp.setControlKeepAliveTimeout(300);
            boolean isFileDownloaded = false;
            try{
                isFileDownloaded = ftp.retrieveFile(remote, output);

            }
            catch(IOException ex){
                if(ftp.isConnected()){

                    try{
                        int retryCode = 0;
                        while (!String.valueOf(retryCode).startsWith("2") && (new Date().getTime() < (startTime_ms + TOTAL_DURATION_MSECS))){
                            try{

                                 retryCode = ftp.retr(local);
                            }catch(Exception e1){
                                Thread.sleep(1000);
                                System.out.println("File not downloaded !! " + e1.toString());
                                ftpConnect(server, ftp, port, startTime_ms);
                            }

                        }
                    }catch(Exception e){
                        //ftp.getReplyCode()
                        //throw e;
                        //disconnect(ftp);
                        //ftpConnect(server, ftp, port, startTime_ms);

                        System.out.println("File not downloaded !! " + e.toString());
                    }                   
                }
            }
            if(isFileDownloaded){
                LOGGER.info("Finished downloading " + remote + " from "
                        + server + " on "
                        + (port > 0 ? port : ftp.getDefaultPort()) + " to "
                        + local);
            }
            else{
                System.out.println("File not downloaded !! ");
            }           

        }catch(IOException io){
            io.printStackTrace();

            throw io;
        }           
        finally {
            if (output != null) {
                try {
                    output.close();
                } catch (IOException f) {
                    LOGGER.severe("output.close() error: "
                            + ServiceUtils.stackToString(f));
                }
            }
        }
    }       
    catch (FTPConnectionClosedException e) {
        LOGGER.severe("Server closed connection: " + server + " "
                + ServiceUtils.stackToString(e));
        throw e;
    } catch (IOException e) {
        LOGGER.severe("IOException, server: " + server + " "
                + ServiceUtils.stackToString(e));
        throw e;
    } finally {
        if (loggedIn) {
            try {
                ftp.logout();
            } catch (IOException f) {
                LOGGER.severe("ftp.logout() error: "
                        + ServiceUtils.stackToString(f));
            }
        }
        disconnect(ftp);
    }
}

我的问题基本上是如果retrieveFile()方法失败,无论如何我可以重新连接并从断开连接点开始下载。

现在我使用的是Microsoft IIS服务器作为我的FTP服务器,但在生产环境中,它将是一个FileZilla FTP服务器。

感谢任何帮助。 感谢

2 个答案:

答案 0 :(得分:1)

很抱歉很晚才回复。这就是我为解决问题所做的工作:

  1. 设置FileZilla服务器
  2. 使用retrieveFileStream而不是retrieveFile。
  3. 保持写入的字节数并使用相同的输出流进行写入。

        BufferedInputStream inputStream = null;
        BufferedOutputStream outputStream = null;
        InputStream in = null;
        int osCount = 0;
        in = ftp.retrieveFileStream(remote);
                inputStream = new BufferedInputStream(in);
                File localFile = new File(local, fileName);
                outputStream = new BufferedOutputStream(new FileOutputStream(localFile));
                for (int read = inputStream.read(); read != -1; read = inputStream.read()) {
                    outputStream.write(read);
                    osCount++;
                }           
                outputStream.flush();
                if (osCount < fileSize) {                       
                    System.out.println(fileName + ": Errored out !!!");
                    BufferedInputStream inputStream1 = null;
                    InputStream in1 = null;
                    if (ftp.isConnected()) {
                        try {
                            while (osCount < fileSize && new Date().getTime() < (startTime_ms + TOTAL_DURATION_MSECS)) {
                                try {                                       
                                        disconnect(ftp);
                                        ftpConnect(server, ftp, port, startTime_ms);
                                        ftp.login(username, password);
                                        ftp.setRestartOffset(osCount);                                      
                                        System.out.println(fileName + ": Re reading from position " + osCount);
                                        in1 = ftp.retrieveFileStream(remote);
                                        inputStream1 = new BufferedInputStream(in1);
                                        for (int read = inputStream1.read(); read != -1; read = inputStream1.read()) {
                                            outputStream.write(read);
                                            osCount++;
                                        }
                                        outputStream.flush();
                                        if(FTPReply.isPositiveCompletion(ftp.getReplyCode())){
                                            isErrored = true;
                                            break;
                                        }
                                } catch (Exception e1) {
                                    Thread.sleep(1000);
                                    LOGGER.severe(fileName + ": File not downloaded " + server + " " + ServiceUtils.stackToString(e1));
                                }
                            }
    
                        } catch (Exception e) {
                            LOGGER.severe(fileName + ": File not downloaded " + server + " " + ServiceUtils.stackToString(e));
                        }
                    }
                }
    

答案 1 :(得分:0)

FTPCLient.restart(...)protected。我认为setRestartOffset(...)retrieveFile(...)是可行的方法:“注意:如果您使用了setRestartOffset(long),则文件数据将从选定的偏移量开始。”

我还会捕获CopyStreamException而不只是IOException“CopyStreamException允许您确定传输的字节数”'。

在进入重试块之前检查ftp.isConnected()但是您编写“FTP服务器松散的网络连接并且可能保持断开连接”。我想如果你输入这个catch块连接已经丢失了,这就是进入这个块的原因之一。

ftp.retr(...)在做什么?重试,是的,但它是否按预期工作?

如果您通过以下方式调试代码,您会得到什么:

  • ftp.isConnected()
  • 设置断点
  • 检索足够大的文件以便下载
  • 在传输过程中拔出网络电缆
  • 查看ftp.isConnected()
  • 的值
  • 如果是true(不太可能,但谁知道)ftp.retr(...)会返回什么?