python文件写入线程不写所有行

时间:2015-02-09 13:36:20

标签: python multithreading

我测试了我的代码输出并对其进行排序,但我没有按预期进行0-1999,缺少行。我的代码线程不安全吗?请建议如何在我的代码中添加线程锁,我发现我的除了所有不会抛出任何错误,是否正确?感谢

import time, threading, random

class MyThreadWithArgs(threading.Thread):
    def __init__(self, string):
        threading.Thread.__init__(self)
        self.string = string

    def run(self):
        try:
            fo = open("foo.np2", "a")
            fo.write( self.string )
            fo.write( '\n' )
            time.sleep(random.uniform(0.1, 0.9))
            fo.close()
        except:
            print ("error logging " + self.string)

ttl_threads = 2000
for i in range(ttl_threads):
    t = MyThreadWithArgs(string = str(i))
    t.start()

2 个答案:

答案 0 :(得分:2)

这可能会按预期工作。为简单起见,删除了异常处理。

import time, threading, random

class MyThreadWithArgs(threading.Thread):
    def __init__(self, string):
        threading.Thread.__init__(self)
        self.string = string

    def run(self):
        fo.write( self.string + '\n' )
        time.sleep(random.uniform(0.1, 0.9))

fo = open("foo.np2", "a")
ttl_threads = 2000
for i in range(ttl_threads):
    t = MyThreadWithArgs(string = str(i))
    t.start()

在原始程序中,您将在追加模式下打开同一文件的多个句柄。每个句柄都维护自己的指针,指向它认为是文件末尾的内容,但是线程0可以在thread-1写入之前修改文件。当调用open时,thread-1仍将写入文件结尾 WAS 的位置。

通过仅保留一个文件描述符,您只有一个文件结束指针,并且基础write系统调用可能可通过操作系统内部锁定在给定文件描述符上重入机制。

我做的另一个更改是将两个调用中的字符串连接到write(),因为作为两个单独的调用,您为调度提供了在系统调用之间切换线程的机会,并可能最终得到连续两个self.string值后跟一行中的两个或多个\n字符串。

我不知道python对write提出什么(如果有的话)保证,我只是想知道<unistd.h> write()在大多数POSIX上如何在C中工作平台。如果您需要保证,请查看python文档,或使用锁定围绕write()电话。

答案 1 :(得分:0)

import time, sys, threading, random

class MyThreadWithArgs(threading.Thread):
    def __init__(self, i, global_msg_i):
        threading.Thread.__init__(self)
        self.i = i
        self.global_msg_i = global_msg_i
    def run(self):
        global global_msg
        try:
            i = self.i
            global_msg_i = self.global_msg_i
            time.sleep(random.uniform(0.1, 0.9))
            print (i)
            global_msg[global_msg_i] = str(i)
            time.sleep(random.uniform(0.1, 0.9))
        except:
            pass
        finally:
            pass

def reset_threads_global_msg():
    global threads, global_msg, global_msg_i, ttl_threads
    threads = []
    global_msg = [None] * ttl_threads
    global_msg_i = 0

def start_join_threads_write_global_msg():
    global threads, global_msg
    for x in threads:
        x.start()
        time.sleep(0.001) # avoid thread create error when creating too fast?!
    [x.join() for x in threads]
    fo = open("foo_test.np++", "a")
    for msg in global_msg:
        if msg is not None:
            fo.write( msg + '\n')
    fo.close()

ttl_threads = 200
reset_threads_global_msg()
for ttl_threads_i in range(1, ttl_threads + 1):
    t = MyThreadWithArgs(i = ttl_threads_i, global_msg_i = global_msg_i)
    threads.append(t)
    global_msg_i += 1
start_join_threads_write_global_msg()
reset_threads_global_msg()