Question

我是python的新手，我从我遇到的教程中得到了这个多线程。不确定是否是粘性练习。

我要存档的内容： ping主机名列表并向上或向下返回。将结果写入csv文件

此文件目前的作用是： ping主机名列表并向上或向下返回。它创建的csv文件是空的，似乎没有向它写任何结果。

我已经完成了一些测试，发现对于我而言，使用串行代码的多线程读取速度大约快16倍。我正在进行大约9000次的ping操作，并希望它们尽快返回。

请告诉我csv部分出错的地方。

import threading
from queue import Queue
import time
import subprocess as sp
import csv

# lock to serialize console output
lock = threading.Lock()

def do_work(item):
    #time.sleep(1) # pretend to do some lengthy work.
    # Make sure the whole print completes or threads can mix up output in one line.

    status,result = sp.getstatusoutput("ping -n 3 " + str(item))
    if status == 0:
        result = 'Up'
    else:
        result = 'Down'

    with lock:
        output.writerow({'hostname': item,'status': result})
        array.append({'hostname': item,'status': result})
        print(threading.current_thread().name,item,result)

# The worker thread pulls an item from the queue and processes it
def worker():
    while True:
        item = q.get()
        do_work(item)
        q.task_done()

# Create the queue and thread pool.
q = Queue()
for i in range(100):
     t = threading.Thread(target=worker)
     t.daemon = True  # thread dies when main thread (only non-daemon thread) exits.
     t.start()

array = []
# stuff work items on the queue (in this case, just a number).
start = time.perf_counter()

headers = ['status','hostname']
output = csv.DictWriter(open('host-export.csv','w'), delimiter=',', lineterminator='\n', fieldnames=headers)
output.writeheader()

txt = open("hosts.txt", 'r', encoding="utf8")
for line in txt:
    q.put(line,array)

q.join()       # block until all tasks are done

# "Work" took .1 seconds per task.
# 20 tasks serially would be 2 seconds.
# With 4 threads should be about .5 seconds (contrived because non-CPU intensive "work")
print(array)

print('time:',time.perf_counter() - start)

我还为csv添加了批量写作思考，也许我只是无法访问函数中的csv对象，但这也没有工作如下。

headers = ['status','hostname']
output = csv.DictWriter(open('host-export.csv','w'), delimiter=',', lineterminator='\n', fieldnames=headers)
output.writeheader()
output.writerows(array)

Answer 1

我弄清楚我做错了什么。我没有关闭文件连接，所以它没有写入文件。

这是我现在用来代替我的csv文件的代码。

fieldnames = ['ip', 'dns', 'pings'] #headings 
test_file = open('test2-p.csv','w', newline='') #open file
csvwriter = csv.DictWriter(test_file, delimiter=',', fieldnames=fieldnames) #set csv writing settings
csvwriter.writeheader() #write csv headings 
for row in rows: #write to csv file
     csvwriter.writerow(row)
test_file.close()

python 3多线程输出到CSV文件（空白）

1 个答案: