防止BufferedReader关闭文件列表中的文件

时间:2011-12-25 19:58:43

标签: python bufferedreader

我有一个扩展BufferedReader的类和一个文件流列表 b.close()被调用除了最后一个流之外的所有流,我想保持流打开 我该怎么做?

感谢


class TestReader(BufferedReader):
    pass

def test(streams):
    for stream in streams:
        b=TestReader(stream)
        do_something(b)
    #all the streams except streams[-1] are closed, how do I prevent this?

streams=[open('test1.txt','rb'),open('test2.txt','rb')]
test(streams)
streams.do_something_else()

1 个答案:

答案 0 :(得分:4)

即使在实现BufferedIOBase类中包装了一个IOBase对象,它们的接口也是一个流(所有继承自IOBase),所以普通行为IOBase对象的一个​​是当它们超出范围时关闭它们自己。 BufferedIOBase只是将close()调用委托给基础流。

您不应将BufferedReader视为流包装器(尽管它是如何实现的),而是作为现有流的类型转换。状态两个流完全绑定在一起。但是,您可以使用detach()取消绑定包装的流,但这会使BufferedIOBase对象无效。

此外,当模式为io.open时,BufferedReader已经返回rb,因此您将进行双缓冲。您应该使用io.FileIO代替。

您有几个选择:

  1. 创建新流和新的基础文件描述符,并传递文件名而不是流。这是您最简单,最安全的选择。

  2. 根据需要创建原始文件描述符并从中创建流。这需要一些 注意多个流不同时使用相同的文件描述符。例如:

    fd = os.open('test.txt', os.O_RDONLY)
    file1 = FileIO(fd, 'r', closefd=False)
    file2 = FileIO(fd, 'r', closefd=False)
    
    file1.read(100)
    assert file1.tell() == 100
    file2.read(100)
    assert file1.tell() == 200
    
  3. detach() BufferedIOBase对象关闭其流之前的基础流。 (记得回流一下!)

    def test(streams):
        for stream in streams:
            b=TestReader(stream)
            do_something(b)
            wrappedstream = b.detach()
            assert wrappedstream is stream
    

    您甚至可以在析构函数中实现它:

    class TestReader(BufferedReader):
        def __del__(self):
            self.detach()
            # self.raw will not be closed,
            # rather left in the state it was in at detachment
    

    如果您认为语义错误,则完全禁用close()委派:

    class TestReader(BufferedReader):
        def close(self):
            self.closed = True
    
  4. 我不了解你在做什么(可能你需要一个不同的设计),但这就是我实现我看到的代码的方式:

    from io import FileIO, BufferedReader
    import io
    import os
    
    class TestReader(BufferedReader):
        pass
    
    def test(streams):
        for stream in streams:
            b = TestReader(stream)
    
    def test_reset(streams):
        """Will try to leave stream state unchanged"""
        for stream in streams:
            pos = stream.tell()
            b = TestReader(stream)
            do_something(b)
            b.detach()
            stream.seek(pos)
    
    
    
    filenames = ['test1.txt', 'test2.txt']
    
    # option 1: just make new streams
    
    streams = [FileIO(name, 'r') for name in filenames]
    test(streams)
    streams = [io.open(name, 'rb') for name in filenames]
    #etc
    
    
    # option 2: use file descriptors
    fds = [os.open(name, os.O_RDONLY) for name in filenames]
    #closefd = False means "do not close fd on __del__ or __exit__"
    #this is only an option when you pass a fd instead of a file name
    streams = [FileIO(fd, 'r', closefd=False) for fd in fds]
    test(streams)
    streams = []
    for fd in fds:
        os.lseek(fd, 0, os.SEEK_SET)
        streams.append(io.open(fd, 'rb', closefd=False))
        # you can also .seek(0) on the BufferedReader objects
        # instead of os.lseek on the fds
    
    
    # option 3: detach
    
    streams = [FileIO(name, 'r') for name in filenames]
    test_reset(streams)
    # streams[*] should still be in the same state as when you passed it in