Question

有没有办法将FTP服务器中存在的csv数据提取到服务器上的MySql Db？我的供应商将数据转储到ftp服务器上的csv文件中。任何帮助将不胜感激。感谢

Answer 1

你问你是否可以FTP到mysql？否。

但您可以在FTP服务器（或可以从FTP服务器检索文件的其他服务器）上启动一个cron作业，然后在脚本中处理它们。

Answer 2

这是一个可能的解决方案，使用bash shell脚本，将其添加到crontab以自动运行。

#!/bin/bash

LOCAL_CSV_PATH="/path/to/local/csv/destination/"
CSV_FILE_NAME="file.csv"

#
# Lets Get This File - delete this section if your csv is local to yoru
#
FTP_HOST='127.0.0.1'
FTP_USER='user@example.com'
FTP_PASS='pass'
FTP_FILE="/path/to/$CSV_FILE_NAME"

# Call 1. Uses the ftp command with the -inv switches.  
#     -i turns off interactive prompting. 
#     -n Restrains FTP from attempting the auto-login feature. 
#     -v enables verbose and progress.  

ftp -inv $FTP_HOST << EOF
user $FTP_USER $FTP_PASS
cd $LOCAL_CSV_PATH
mget $FTP_FILE
delete $FTP_FILE
bye
EOF


#
# Import this file
#

MYSQL_USERNAME=user
MYSQL_PASS=pass

# IFS is the internal field seperator used by bash, set to "," for csv files
IFS=,
while read column1 column2 column3
      do
        echo "INSERT INTO my_table (column1,column2,column3) VALUES ('$column1', '$column2', '$column3');"
      done < "$LOCAL_CSV_PATH/$CSV_FILE_NAME" | mysql -u $MYSQL_USERNAME -p $MYSQL_PASS my_database;

# Cleanup
mv "$LOCAL_CSV_PATH/$CSV_FILE_NAME" "$LOCAL_CSV_PATH/$CSV_FILE_NAME.old_run"

Answer 3

在下面找到从FTP服务器读取文件的Java代码，并将所有文件数据存储到数据库中，每个文件驻留在FTP目录路径中。

import java.sql.Statement;
import java.sql.Connection;
import java.sql.DriverManager;
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.net.ftp.FTPReply;

public class ftpRead {

    public static String sDir = "";
    public static FTPClient ftpClient = new FTPClient();
    public static int replyCode;
    public static boolean success = false;
    public static String server = "ftp server ip";
    //int port = ;
    public static String user = "ftp username";
    public static String pass = "ftp password";

    public static void showServerReply(FTPClient ftpClient) {
        String[] replies = ftpClient.getReplyStrings();
        if (replies != null && replies.length > 0) {
            for (String aReply : replies) {
                System.out.println("SERVER: " + aReply);
            }
        }
    }

    public static void main(String[] agrs) throws IOException {

        try {

            ftpClient.connect(server);
            showServerReply(ftpClient);
            replyCode = ftpClient.getReplyCode();
            if (!FTPReply.isPositiveCompletion(replyCode)) {
                System.out.println("Operation failed. Server reply code: " + replyCode);
                //return "false";
            }

            success = ftpClient.login(user, pass);
            showServerReply(ftpClient);
            if (!success) {
                System.out.println("Could not login to the server");
                // return "false";
            } else {
                System.out.println("User successfully logged in.");
                ftpClient.changeWorkingDirectory("ftp_file_directory");//directory Changed

                FTPFile[] files = ftpClient.listFiles();//get list of files and folders

                for (int ii = 0; ii < files.length; ii++) {

                    InputStream fstream = null;
                    BufferedReader br = null;
                    DataInputStream in = null;

                    if (files[ii].isDirectory()) {//check for file or directory


                        System.out.println("Directory name  " + files[ii].getName());//print directory name

                    } else {

                        System.out.println("file name  " + files[ii].getName());//print file name
                        //  try {
                        ftpClient.connect(server);
                        showServerReply(ftpClient);
                        replyCode = ftpClient.getReplyCode();
                        if (!FTPReply.isPositiveCompletion(replyCode)) {
                            System.out.println("Operation failed. Server reply code: " + replyCode);
                            //return "false";
                        }
                        success = ftpClient.login(user, pass);

                        showServerReply(ftpClient);
                        ftpClient.changeWorkingDirectory("path_ to_file_directory");
                        fstream = ftpClient.retrieveFileStream(files[ii].getName());

                        in = new DataInputStream(fstream);
                        br = new BufferedReader(new InputStreamReader(in));
                        String strLine = null;
     while ((strLine = br.readLine()) != null) {
                        String dataline = strLine;
                        String insertBackup = "insert into tablename values(" + dataline + ")";
                        exeQuery(insertBackup);


                        if (strLine.trim().length() == 0) {
                            continue;  // Skip blank lines
                        }

                    }

                    }
                }


            }

        } catch (IOException ex) {
            System.out.println("Oops! Something wrong happened");
        }

    }

    public static void exeQuery(String query) {
        Connection conn = null;
        String url = "jdbc:mysql://localhost:3306/";

        String driver = "com.mysql.jdbc.Driver";
        String userName = "mysqluser";
        String password = "mysqlpass";
        try {
            Class.forName(driver).newInstance();
            conn = DriverManager.getConnection(url + "database_name", userName, password);
            System.out.println("Connected to the database");
            Statement stmt = null;
            stmt = conn.createStatement();
            stmt.executeUpdate(query);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

从以下链接下载所有支持库。

FTP Library

MySQL connector

注意：请确保根据您的FTP凭据，数据库凭据等对此代码进行疯狂更改。

使用ftp服务器CSV数据填充MySql数据库

3 个答案: