如何在Python中以相反的顺序读取CSV文件?

时间:2012-06-07 14:28:25

标签: python csv

我知道如何为TXT文件执行此操作,但现在我在为CSV文件执行操作时遇到了一些麻烦。

如何在Python中从底部读取CSV文件?

3 个答案:

答案 0 :(得分:20)

与文本文件几乎相同:将整个内容读入列表然后向后:

import csv
with open('test.csv', 'r') as textfile:
    for row in reversed(list(csv.reader(textfile))):
        print ', '.join(row)

如果你想得到花哨的话,你可以编写很多代码来读取从文件末尾开始并向后工作的块,一次发出一行,然后将其提供给csv.reader,但是只能用于可以搜索的文件,即磁盘文件,但不能用于标准输入。


  

我们中的一些人的文件不适合内存,任何人都可以使用不需要将整个文件存储在内存中的解决方案吗?

这有点棘手。幸运的是,所有csv.reader期望的是一个类似迭代器的对象,每次调用next()时返回一个字符串(行)。所以我们抓住Darius Bacon在“Most efficient way to search the last x lines of a file in python”中提出的技术来向后读取文件的行,而不必拉入整个文件:

import os

def reversed_lines(file):
    "Generate the lines of file in reverse order."
    part = ''
    for block in reversed_blocks(file):
        for c in reversed(block):
            if c == '\n' and part:
                yield part[::-1]
                part = ''
            part += c
    if part: yield part[::-1]

def reversed_blocks(file, blocksize=4096):
    "Generate blocks of file's contents in reverse order."
    file.seek(0, os.SEEK_END)
    here = file.tell()
    while 0 < here:
        delta = min(blocksize, here)
        here -= delta
        file.seek(here, os.SEEK_SET)
        yield file.read(delta)

并将reversed_lines提供给代码,以便在到达csv.reader之前反转行,从而无需reversedlist

import csv
with open('test.csv', 'r') as textfile:
    for row in csv.reader(reversed_lines(textfile)):
        print ', '.join(row)

可能有更多的Pythonic解决方案,它不需要在内存中逐个字符地反转块(提示:只需获取块中有行结束的索引列表,反转它,以及使用它来切片块),并使用chain中的itertools将连续块中的线簇粘合在一起,但这仍然是读者的练习。


  

值得注意的是,只有CSV文件中的列不包含换行符时,上面的reversed_lines()惯用法才有效。

AARGH!总有一些东西。幸运的是,解决这个问题并不算太糟糕:

def reversed_lines(file):
    "Generate the lines of file in reverse order."
    part = ''
    quoting = False
    for block in reversed_blocks(file):
        for c in reversed(block):
            if c == '"':
                quoting = not quoting
            elif c == '\n' and part and not quoting:
                yield part[::-1]
                part = ''
            part += c
    if part: yield part[::-1]

当然,如果您的CSV方言不使用",则需要更改引号字符。

答案 1 :(得分:0)

Building on @mike-desimone 's answer. Here's a solution that provides the same structure as a python file object but is read in reverse, line by line:

import os

class ReversedFile(object):
    def __init__(self, f, mode='r'):
        """
        Wraps a file object with methods that make it be read in reverse line-by-line

        if ``f`` is a filename opens a new file object

        """
        if mode != 'r':
            raise ValueError("ReversedFile only supports read mode (mode='r')")

        if not type(f) == file:
            # likely a filename
            f = open(f)

        self.file = f
        self.lines = self._reversed_lines()

    def _reversed_lines(self):
        "Generate the lines of file in reverse order."
        part = ''
        for block in self._reversed_blocks():
            for c in reversed(block):
                if c == '\n' and part:
                    yield part[::-1]
                    part = ''
                part += c
        if part: yield part[::-1]

    def _reversed_blocks(self, blocksize=4096):
        "Generate blocks of file's contents in reverse order."
        file = self.file

        file.seek(0, os.SEEK_END)
        here = file.tell()
        while 0 < here:
            delta = min(blocksize, here)
            here -= delta
            file.seek(here, os.SEEK_SET)
            yield file.read(delta)


    def __getattribute__(self, name):
        """ 
        Allows for the underlying file attributes to come through

        """ 
        try:
            # ReversedFile attribute
            return super(ReversedFile, self).__getattribute__(name)
        except AttributeError:
            # self.file attribute
            return getattr(self.file, name)

    def __iter__(self):
        """ 
        Creates iterator

        """ 
        return self

    def seek(self):
        raise NotImplementedError('ReversedFile does not support seek')

    def next(self):
        """
        Next item in the sequence

        """
        return self.lines.next()

    def read(self):
        """
        Returns the entire contents of the file reversed line by line

        """
        contents = ''

        for line in self:
            contents += line

        return contents

    def readline(self):
        """
        Returns the next line from the bottom

        """
        return self.next()

    def readlines(self):
        """
        Returns all remaining lines from the bottom of the file in reverse

        """
        return [x for x in self]

答案 2 :(得分:0)

去吧。这是一个简单的程序来反转CSV文件中的行。

import csv
BC_file = open('Master.csv', 'rb')
BC_reader = csv.reader(BC_file)
next(BC_reader)
for row in reversed(list(BC_reader)):
    print row[0]