Question

我正在尝试使用python字典中的cStringIO读取一个非常大的字符串流：

def stream_read(self, path):
    try:
        # create a string stream from the contents at 'path'
        # note: the string at self._my_dict[path] is 7MB in size
        stream = StringIO.StringIO(self._my_dict[path])
        while True:
            # buffer size is 128kB, or 128 * 1024
            buf = stream.read(self.buffer_size)
            if buf != '':
                yield buf
            else:
                raise StopIteration
    except KeyError:
        raise IOError("Could not get content")

在我的测试套件中，我通过首先测试stream_write，断言数据存在于该路径，然后调用stream_read来测试此函数：

def test_stream(self):
    filename = self.gen_random_string()
    # test 7MB
    content = self.gen_random_string(7 * 1024 * 1024)
    # test stream write
    io = StringIO.StringIO(content)
    self._storage.stream_write(filename, io)
    io.close()
    self.assertTrue(self._storage.exists(filename))
    # test read / write
    data = ''
    for buf in self._storage.stream_read(filename):
        data += buf
    self.assertEqual(content, data)

然而在我的测试套件中，我正在捕获AssertionError：

======================================================================
FAIL: test_stream (test_swift_storage.TestSwiftStorage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bacongobbler/.../test/test_local_storage.py", line 44, in test_stream
    self.assertEqual(content, data)
AssertionError: '[squelched]' != '<cStringIO.StringI object at 0x3148e70>'
----------------------------------------------------------------------
Ran 28 tests in 20.495s

FAILED (failures=1)

它看起来与an issue I posted last week有关，但我仍然不太清楚为什么stream在这种情况下将{{1}}设置为生成器作为字符串。

如果有人想仔细查看源代码，那么全部都在https://github.com/bacongobbler/docker-registry/blob/106-swift-storage/test/utils/mock_swift_storage.py

Answer 1

在调用StringIO时，您只需 self._storage.stream_write(filename, io)对象：

def put_content(self, path, content, chunk=None):
    path = self._init_path(path)
    try:
        self._swift_container[path] = content
    except Exception:
        raise IOError("Could not put content")

其中content是您传入的io对象。

稍后，您再次将该文件对象传递给StringIO：

stream = StringIO.StringIO(self.get_content(path))

这会在str()上调用self.get_content(path)，存储cStringIO.StringI()实例的字符串表示形式：

>>> from cStringIO import StringIO
>>> str(StringIO('test data'))
'<cStringIO.StringI object at 0x1074ea470>'

您的阅读代码运行正常，您的编写模拟需要实际获取StringIO对象的 out 数据。

.read()来电将在此处进行：

def put_content(self, path, content, chunk=None):
    path = self._init_path(path)
    try:
        self._swift_container[path] = content.read()
    except Exception:
        raise IOError("Could not put content")

在python中使用generator和cStringIO来传输字符串

1 个答案: