使用python将远程计算机上运行的配置单元查询的输出管道传输到本地计算机

Question

所有

我尝试使用ssh协议（远程地址为xx.xxx.xxx.51）在远程计算机上运行hive命令（在表上运行查询），然后将csv文件传递给本地机器使用python和pxssh（pexpect）。我也使用subprocess和paramiko。

1） Code1使用pxssh

from pexpect import pxssh
import getpass
try:
    s = pxssh.pxssh()
    hostname, username, password = ('xx.xxx.xxx.51', 'username', 'passwd')
    s.login(hostname, username, password)
    cmd = "hive -e 'use dbname; set hive.resultset.use.unique.column.names=false ; select * from tablename limit 50'   >  ./myfile.csv"
    print cmd
    s.sendline(cmd) 
    s.prompt()             # match the prompt
    print(s.before)        # print everything before the prompt. 
    s.logout()
except pxssh.ExceptionPxssh as e:
    print("pxssh failed on login.")
    print(e)

错误：它在远程服务器上输出文件myfile.csv，但在本地计算机上没有。我希望将csv文件保存在运行python脚本的本地机器上。

实际的命令是：

hive -e 'use dbname; set hive.resultset.use.unique.column.names=false ; select * from tablename limit 50' | sed 's/[\t]/,/g '  >  ./myfile.csv"

但是选项| sed 's/[\t]/,/g会出错。

2）使用子流程的解决方案

import subprocess

command = "ssh username@xx.xxx.xxx.51  ls -tal "   # an example using ls -tal
print "submitting command", command
result = subprocess.Popen(command, shell = True  stdout=subprocess.PIPE, stdin=subprocess.PIPE)
result.stdin.write('passwd\n')
result.stdin.flush()
print "got response"
response,err = result.communicate()
print response

3）使用paramiko解决方案

import paramiko

#Configuration for remote machine

server, username, password = ('xx.xxx.xxx.51', 'username', 'passwd')
ssh1 = paramiko.SSHClient()
ssh1.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh1.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))
#Loads the user's local known host file.
ssh1.connect(server, username=username, password=password)

# doing config for local machine
server2, username2, password2 = ('yy.yyy.yyy.112', 'username', 'passwd2')
ssh2 = paramiko.SSHClient()
ssh2.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh2.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))
ssh2.connect(server2, username=username2, password=password2)

_, stdout2, _ = ssh2.exec_command('hostname')
hostname2 = stdout2.read()

command = " hive -e 'use dbname; set hive.resultset.use.unique.column.names=false ; \
            select * from tablename limit 20' |   ssh  username@yy.yyy.yyy.112 'cat > myfile.csv' " 


#print cmd
ssh_stdin, ssh_stdout, ssh_stderr = ssh1.exec_command(command)
print "output", ssh_stdout.read() #Reading output of the executed command
error = ssh_stderr.read()
#Reading the error stream of the executed command
print "err", error, len(error) 

这会产生以下错误输出：

output 
err Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

如何将远程服务器上运行的配置单元查询输出的结果传输到本地计算机？