所有我尝试使用ftplib(jenkins作业)将文件从远程服务器复制到另一台服务器时遇到以下错误。在我的本地系统中,它运行正常
1)还提到了python代码,有人可以让我知道如何解决这个问题吗?
2)如果有人可以指导如何提及服务器,用户名和密码而无需在脚本中进行硬编码,这也将非常有帮助。
错误:
12:33:00 [EnvInject] - Loading node environment variables.
12:33:00 Building in workspace /var/lib/jenkins/jobs/log_porting_from_magento/workspace
12:33:00 [workspace] $ /bin/sh -xe /tmp/jenkins34559567899962487.sh
12:33:00 + export PYTHONPATH=/home/etl/bi/
12:33:00 + cd /home/etl/bi/crm
12:33:00 + python3 -u log_porting.py
12:33:00 DWH Connected
12:33:00 Table staging.stock_dump Created at DWH
12:33:01 FTP connection succesful
12:33:01 Error occured while copying file from source to destination
12:33:01 Error: [Errno 13] Permission denied: '/home/etl/bi/csv_imports/log'
12:33:01 Finished: SUCCESS
下面是python代码,
from psycopg2 import connect, sql
from time import perf_counter
import psycopg2
import time
import os
from ftplib import FTP
start_time = time.perf_counter()
cnx_psql = psycopg2.connect(host="localhost", database="postgres", user="postgres",
password="postgres", port="5432")
# Cursors initializations
cur_psql = cnx_psql.cursor()
def getFile(ftp, filename):
try:
local_filename = os.path.join(r"/Users/sand/Downloads/", filename)
#local_filename = os.path.join(r"/home/etl/bi/csv_imports/", filename)
lf = open(local_filename, "wb")
ftp.retrbinary("RETR " + filename ,lf.write)
print("file copied")
except (Exception, psycopg2.Error) as error:
print ("Error: {}".format(error))
try:
filePath='''/Users/sand/Downloads/log'''
table='staging.stock_dump'
SQL="""DROP TABLE IF EXISTS """+ table + """;CREATE TABLE """+ table + """
(created_date TEXT, product_sku TEXT, previous_stock TEXT, current_stock TEXT );"""
#keys = ['created_date', 'product_sku', 'previous_stock', 'current_stock']
#sql.SQL("""CREATE TABLE IF NOT EXISTS {} ({});""").format(sql.Identifier(table),sql.SQL(' TEXT,').join(map(sql.Identifier, keys))))
cur_psql.execute(SQL)
cnx_psql.commit()
try:
ftp = FTP('server.com','username','password')
print("FTP connection succesful")
ftp.cwd('/stockitem/')
except (Exception, psycopg2.Error) as error:
print ("Error: {}".format(error))
getFile(ftp,'log')
read_file = open(filePath, "r")
my_file_data = read_file.readlines()
for line in my_file_data:
if 'Stock:' in line:
fields=line.split(" ")
date_part1=fields[0]
date_part2=fields[1][:-1]
sku=fields[3]
prev_stock=fields[5]
current_stock=fields[7]
if prev_stock.strip()==current_stock.strip():
continue
else:
cur_psql.execute("insert into " + table+"(created_date, product_sku, previous_stock , current_stock)" + " select CAST('" + date_part1+ " "+ date_part2 + "' AS TEXT)" +", CAST('"+sku+"' AS TEXT),CAST('" + prev_stock +"' AS TEXT),CAST('" +current_stock + "' AS TEXT);")
cnx_psql.commit()
cur_psql.close()
cnx_psql.close()
print("Data loaded to DWH from text file")
print("Data porting took %s seconds to finish---" % (time.perf_counter() - start_time))
except (Exception, psycopg2.Error) as error:
print ("Error: {}".format(error))
print("Error adding information.")
quit()
finally:
ftp.close()