Question

所以，我在这里：我正在用Python编写一个执行以下操作的脚本：

启动命令以通过SSH在服务器上启动日志记录过程（bash脚本）
使用ab从客户端对服务器进行基准测试
使用scp恢复在服务器上生成的日志文件
使用matplotlib生成带有客户端数据的图形

我将在最后附上两个脚本，但是我怀疑它们对这个问题是否有用。

为了不等待通过SSH启动的日志记录脚本结束以继续python脚本，我将-f与SSH一起使用在后台发送它。 python脚本确实还在继续，但是我发现自己遇到了与日志记录过程和基准测试时间有关的问题，具体取决于客户端的网络访问和处理能力，该问题可能会大为不同。

我希望以一种允许我自己继续使用客户端上的python脚本执行基准的方式在服务器上启动日志记录命令，但是我也不想在日志记录之前启动绘图客户端完成服务器端。现在，我无法通过计算大概的睡眠时间来做到这一点，因为睡眠时间会因客户端和连接而异。

因此，我想仅在客户端上退出日志记录脚本时才开始检索日志文件并进行绘图，这将是SSH命令的成功执行和退出。

我如何跟踪python中的SSH命令状态，以仅在退出SSH命令（在后台）后才启动脚本的那一部分？

我想避免使用线程，因为它不太适合脚本。我当然愿意接受我没有的任何更好的建议。

服务器端日志文件

#!/usr/bin/env bash
#perf_measurement.sh

# logs the load average every n seconds for a y period of time in a csv file

# interval, in seconds, for the load measure
INTERVAL=1
# amount of measures
MEASURES=$1
# path of the logging file
LOGFILE_EXT=".csv"
LOGFILE="$2""$LOGFILE_EXT"
# separator for the csv file
SEPARATOR=","
# load average file
LOADFILE="/proc/loadavg"
# timestamp at start of measure/script
INIT_TIMESTAMP=`date +%s`

# clearing logfile
if [ -e $LOGFILE ]; then
  rm $LOGFILE
fi

# function to get one of the load average values from LOADFILE
loadpm () {
  cat $LOADFILE | cut -d' ' -f$1
}
# function generation a timestamp starting from the start of the script, in seconds
timestamp () {
  echo $(expr $(date +%s) - $INIT_TIMESTAMP)
}

for (( i=0; i<$MEASURES; i++ )) ; do
  echo $(timestamp)$SEPARATOR$(loadpm 1) >> $LOGFILE
  sleep $INTERVAL
done


exit 0

Python脚本，客户端 ＃！/ usr / bin / python ＃benchmark.py

import csv
import matplotlib.pyplot as plt
import os
import time

SERVER = "edited"
SERVER_USER = "ubuntu"
SERVER_PERF_MEASUREMENT_SCRIPT_PATH = "/var/www/html"
SERVER_PERF_MEASUREMENT_SCRIPT = "perf_measurement.sh"
REQUESTS = 1000000 # high because time limitation is used instead of number of requests to facilitate logging

CONCURRENCY = input("Input concurrency for test : ")
MEASURE_TIME = input("Input time of the test (in seconds) : ")
FILE = input("Input name of file/repo for test (without the extension) : ")
TEST_NAME = input("Input a name for the test (no space) : ")
FILENAME= f"{TEST_NAME}_conc_{CONCURRENCY}"
GRAPH_FILE = f"{FILENAME}.png"
LOG_FILE = f"{FILENAME}.csv"

FILE_PATH = f"/{FILE}"
TEST_TIME = int(int(MEASURE_TIME) - int(MEASURE_TIME) / 4)
SLEEP_TIME = int(int(MEASURE_TIME) / 8)


SSH_COMMAND = f"ssh -f {SERVER_USER}@{SERVER} 'cd {SERVER_PERF_MEASUREMENT_SCRIPT_PATH}&&./"\
                f"{SERVER_PERF_MEASUREMENT_SCRIPT} {MEASURE_TIME} {FILENAME}'"
AB_COMMAND = f"ab -c {CONCURRENCY} -t {TEST_TIME} -n {REQUESTS} {SERVER}{FILE_PATH}"
SCP_COMMAND = f"scp {SERVER_USER}@{SERVER}:{SERVER_PERF_MEASUREMENT_SCRIPT_PATH}/{LOG_FILE} ."

print(SSH_COMMAND)

print("\nStarting the logging server side...\n")
os.system(SSH_COMMAND)

print(f"\nSleeping {SLEEP_TIME} seconds to visualize load gain...\n")
time.sleep(SLEEP_TIME)

print("\nStarting benchmark...\n")
os.system(AB_COMMAND)

print("\nRecovering log file from server...\n")
os.system(SCP_COMMAND)


print("\nGenerating graph...\n")
# declaring list for coordinates of the graph
x_coord = []
y_coord = []

# opening log file
with open(LOG_FILE, "r") as csvfile:

    # reading csv logfile
    logfile = csv.reader(csvfile)

    # iterating through logfile
    for row in logfile:
        # storing coordinates in list
        x_coord.append(int(row[0]))
        y_coord.append(float(row[1]))


# generation graph
plt.plot(x_coord, y_coord)
plt.ylabel('Load average (from /proc/loadavg)')
plt.xlabel('Time')

# exporting graph to png
plt.savefig(GRAPH_FILE)

Answer 1

鉴于此脚本的独特性，它似乎是subprocess.Popen(...)的一个好用例。

您可以使用Popen启动带有进程句柄的命令，而不是将SSH命令发送到后台。然后，您可以继续进行需要进行的任何测试，然后scp返回结果之前，返回该流程句柄。这应该起作用，因为ssh命令在其方法调用的末尾提供了一组命令时，将发送命令而不是打开交互式登录外壳，然后将输出返回到客户端计算机。远程命令完成后，ssh将关闭连接并退出其自己的进程。因此，以下应该起作用：

import subprocess
import shlex

# ... your other imports and code...

_PROC_TIMEOUT_SEC = 5
SSH_COMMAND = (
                f"ssh {SERVER_USER}@{SERVER} 'cd {SERVER_PERF_MEASUREMENT_SCRIPT_PATH}&&./"
                f"{SERVER_PERF_MEASUREMENT_SCRIPT} {MEASURE_TIME} {FILENAME}'"
              )
AB_COMMAND = f"ab -c {CONCURRENCY} -t {TEST_TIME} -n {REQUESTS} {SERVER}{FILE_PATH}"
SCP_COMMAND = f"scp {SERVER_USER}@{SERVER}:{SERVER_PERF_MEASUREMENT_SCRIPT_PATH}/{LOG_FILE} ."

print("\nStarting the logging server side...\n")
portioned_ssh_command = shlex.split(SSH_COMMAND)
ssh_proc = subprocess.Popen(portioned_ssh_command)

# Still perform your testing as you had planned...
# Not sure if you need to sleep here, but I'll leave that up to you.
print(f"\nSleeping {SLEEP_TIME} seconds to visualize load gain...\n")
time.sleep(SLEEP_TIME)
print("\nStarting benchmark...\n")
os.system(AB_COMMAND)

# Wait for SSH process to finish before attempting to scp log file.
ssh_return_code = None
while(ssh_return_code is None):
    try:
        ssh_return_code = ssh_proc.wait(_PROC_TIMEOUT_SEC)
    except subprocess.TimeoutExpired:
        pass

print(f"Info: SSH completed with exitcode {ssh_proc.returncode}")

# Now that SSH 
print("\nRecovering log file from server...\n")
os.system(SCP_COMMAND)

# ... The rest of your code...

注意：我尚未测试上面的代码，但是我非常有信心将完成您希望完成的工作。

脚本中的SSH -f-执行命令时获取信号

1 个答案: