将文件复制到远程服务器,然后将其加载到MySQL数据库

时间:2020-01-12 18:55:09

标签: python mysql ssh ssh-tunnel load-data-infile

我的计算机上有一个文件,该文件会一点一点地更新,我想将此文件上传到远程服务器,然后将其放在同一服务器上的MySQL数据库中。服务器正在运行CENTOS 7,计算机正在运行Windows。已经在Google中检查了多个页面和许多搜索,最后得到了以下代码:

import base64
import paramiko

import pymysql

from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
import pandas as pd


def sendFileFromTo(client, orig, dest):
    sftp = client.open_sftp()
    sftp.put(orig, dest)
    sftp.close()


def exe(client, command):
    stdin, stdout, stderr = client.exec_command(command)
    for line in stdout:
        print('... ' + line.strip('\n'))

    def main():

    client = paramiko.SSHClient()

    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(host, port=port, username='root', password='pass')
    print("Connected")
    #exe(client, 'ls')

    sendFileFromTo(
        client,
        orig,
        dest)

    print("File Sent")

    client.close()

    print("SSH CLOSED")

    with SSHTunnelForwarder(
        (host, port),
        ssh_username='root',
        ssh_password='pass',
        remote_bind_address=('127.0.0.1', 3306),
            local_bind_address=('127.0.0.1', 3305)) as tunnel:

        print("STARTING MYSQL")
        with pymysql.connect(host=tunnel.local_bind_host, user='root',
                             passwd='rootSolar', db='TEST',
                             port=tunnel.local_bind_port, local_infile=True) as conn:

            print("LOADING QUERY")
            #query = '''USE TEST'''
            # print(query)
            #data = pd.read_sql_query(query, conn)
            query = '''LOAD DATA INFILE '/root/copied_file' INTO TABLE RS FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\\n' IGNORE 4 ROWS (@time,record,Radiacion,Radiacion_UVA) SET time = STR_TO_DATE(@time,'%Y-%m-%d %H:%i:%s');'''
            # print(query)
            #data = pd.read_sql_query(query, conn)
            with conn.cursor() as cur:
                cur.executemany(query)
                conn.escape_string()
                conn.commit()


if __name__ == "__main__":
    main()

我现在所拥有的与远程服务器建立了ssh连接,以上传和覆盖文件,关闭文件并创建ssh隧道以连接到MYSQL和数据库。 host和port变量具有用于ssh连接的远程服务器的值,但是由于它是公共地址,因此我不会提供这些详细信息。

该代码可与其他MySQL查询很好地配合,但特别是此程序挂起且终端被冻结。我也不能使用ctrl + c来关闭进程,我必须关闭终端才能关闭它

0 个答案:

没有答案