我有一个列表列表，其中一些子列表完全由None组成，有些列表末尾和字符串之间有None。我需要做三件事：
从子列表的末尾删除None，如果有的话，替换中间的用空字符串分隔的那些。
排除完全None s
使用结果
我的尝试得到了预期的结果，但我想知道是否有更快的方法来做到这一点：
from itertools import islice
rows = [["row 1 index 0",None,"row 1 index 2",None,None],
[None,"row 2 index 1",None,None,None],
[None,None,None,None,None]]
data = []
for r in rows:
for i,c in enumerate(reversed(r)):
if c is not None:
data.append(["" if x is None else
str(x) for x in islice(r,0,len(r)-i)])
break
print (data)
期望的结果/输出：
[['row 1 index 0', '', 'row 1 index 2'], ['', 'row 2 index 1']]
基准（我知道如何）：
from itertools import islice
import time
q = ["string",None,"string",None,"string"] + [None] * 95
rows = [q.copy() for i in range(500000)]
for z in range(1,6):
st = time.time()
data = []
for r in rows:
for i,c in enumerate(reversed(r)):
if c is not None:
data.append(["" if x is None else
str(x) for x in islice(r,0,len(r)-i)])
break
end = time.time()
print ("Run: " + str(z) + "| time: " + str(end-st))
结果（i5 ivybridge windows 10）：
Run: 1| time: 5.787390232086182
Run: 2| time: 5.802111387252808
Run: 3| time: 5.697156190872192
Run: 4| time: 5.38789963722229
Run: 5| time: 5.739344596862793
答案 0 :(得分：3)
您可以使用列表推导逐步消除Nones
from itertools import dropwhile
rows = [["row 1 index 0",None,"row 1 index 2",None,None],
[None,"row 2 index 1",None,None,None],
[None,None,None,None,None]]
# remove lists with all Nones
rows1 = [row for row in rows if set(row) != {None}]
# remove trailing Nones
rows2 = [dropwhile(lambda x: x is None, reversed(row)) for row in rows1]
# replace None with ''
rows3 = [list(reversed([x if x is not None else '' for x in row])) for row in rows2]
print(rows3)
输出：
[['row 1 index 0', '', 'row 1 index 2'], ['', 'row 2 index 1']]
答案 1 :(得分：3)
性能取决于数据的性质！请参阅以下时序，并倾向于您认为更适合您期望遇到的数据集的时序。我提出了一个部分就地解决方案，它现在似乎在我的测试中具有最佳性能，但希望显而易见的是，基准测试需要更多地充实以获得您的权衡的真实情况。
首先我做了一个测试集。
In [60]: rows = [["row 1 index 0",None,"row 1 index 2",None,None],
...: [None,"row 2 index 1",None,None,None],
...: [None,None,None,None,None]]
In [61]: rowsbig = [r*1000 for r in rows]
In [62]: rowsbig = [list(r) for _ in range(1000) for r in rowsbig]
In [63]: sum(len(r) for r in rowsbig)
Out[63]: 15000000
现在，这是一个保持卫生的小帮手：
In [65]: def test_set(source=rowsbig):
...: return [list(r) for r in source]
...:
所以，让我们在函数中包含三个提议的方法：
In [86]: def new_to_coding(rows):
...: data = []
...: for r in rows:
...: for i,c in enumerate(reversed(r)):
...: if c is not None:
...: data.append(["" if x is None else
...: str(x) for x in islice(r,0,len(r)-i)])
...: break
...: return data
...:
In [87]: def Bit(rows):
...: data = [list(map(lambda x: '' if x is None else x, row)) for row in rows]
...: data = [row[:max(i for i, e in enumerate(row, 1) if e is not '')] for row in data if set(row) != {''}]
...: return data
...:
In [88]: def taras(rows):
...: # remove lists with all Nones
...: rows1 = [row for row in rows if set(row) != {None}]
...: # remove trailing Nones
...: rows2 = [dropwhile(lambda x: x is None, reversed(row)) for row in rows1]
...: # replace None with ''
...: rows3 = [list(reversed([x if x is not None else '' for x in row])) for row in rows2]
...: return rows3
...:
快速健全检查：
In [89]: taras(test_set()) == new_to_coding(test_set())
Out[89]: True
In [90]: Bit(test_set()) == new_to_coding(test_set())
Out[90]: True
现在，一些时间设置。 @new_to_coding注意始终使用timeit模块创建基准。天真的time.time()方法忽略了许多细微之处，并且更加方便！
In [91]: from timeit import timeit
In [92]: setup = "from __main__ import new_to_coding, Bit, taras, test_set; testrows = test_set()"
现在，结果：
In [93]: # using OP's method
...: timeit('new_to_coding(testrows)', setup, number=5)
Out[93]: 5.416837869910523
In [94]: # using `Bit`
...: timeit('Bit(testrows)', setup, number=5)
Out[94]: 14.52187539380975
In [95]: # using `taras`
...: timeit('taras(testrows)', setup, number=5)
Out[95]: 3.7361009169835597
所以，似乎增量方法获胜了！当然，数据的确切性质可能会改变这些相对时间。我怀疑＆＃34;所有None＆＃34;的比例行将影响这些方法的相对性能。 警告！结果证明这是真的！见编辑
我已经采用微优化@taras方法，确保所有名称都是函数的本地名称，因此没有全局查找，将list(reversed(alist))替换为alist[::-1]，并制作中间转换生成器表达式，以便只实现单个列表：
In [111]: def is_None(x): return x is None
...:
...: def taras_micro_op(rows, dropwhile=dropwhile, reversed=reversed, set=set, is_None=is_None):
...: # remove lists with all Nones
...: rows1 = (row for row in rows if set(row) != {None})
...: # remove trailing Nones
...: rows2 = (dropwhile(is_None, reversed(row)) for row in rows1)
...: # replace None with ''
...: rows3 = [[x if x is not None else '' for x in row][::-1] for row in rows2]
...: return rows3
...:
In [112]: taras_micro_op(test_set()) == taras(test_set())
Out[112]: True
In [113]: setup = "from __main__ import taras, taras_micro_op, test_set; testrows = test_set()"
In [114]: # using `taras`
...: timeit('taras(testrows)', setup, number=50)
Out[114]: 35.11660181987099
In [115]: # using `taras_micro_op`
...: timeit('taras_micro_op(testrows)', setup, number=50)
Out[115]: 33.70030225184746
In [116]: 33.70030225184746 / 35.11660181987099
Out[116]: 0.9596686611281929
不到5％的改善。实际上，我会抛弃＆＃34;内联默认参数＆＃34;并且只使用中间生成器表达式，如果只是为了帮助提高内存效率。
换句话说，我建议使用以下内容：
In [117]: def taras_memory_op(rows):
...: # remove lists with all Nones
...: rows1 = (row for row in rows if set(row) != {None})
...: # remove trailing Nones
...: rows2 = (dropwhile(lambda x: x is None, reversed(row)) for row in rows1)
...: # replace None with ''
...: rows3 = [[x if x is not None else '' for x in row][::-1] for row in rows2]
...: return rows3
...:
In [118]: setup = "from __main__ import taras, taras_memory_op, test_set; testrows = test_set()"
In [119]: # using `taras`
...: timeit('taras(testrows)', setup, number=50)
Out[119]: 35.10479677491821
In [120]: # using `taras`
...: timeit('taras_memory_op(testrows)', setup, number=50)
Out[120]: 34.00812040804885
In [121]: 34.00812040804885/35.10479677491821
Out[121]: 0.9687599283396816
由于大多数已经很小的改进实际上来自于使用生成器表达式！
所以，我尝试使用op：
提供的测试集In [3]: q = ["string",None,"string",None,"string"] + [None] * 95
...: rows = [q.copy() for i in range(500000)]
...:
In [4]: sum(len(r) for r in rows)
Out[4]: 50000000
注意，在我的原始测试集中，大约有33％＆＃34;所有None＆＃34;行。但是，在上面，没有行都是None 。这肯定会影响性能。
In [7]: def test_set(source=rows):
...: return [list(r) for r in source]
...:
In [8]: setup = "from __main__ import new_to_coding, taras_memory_op, test_set; testrows = test_set()"
In [9]: # using OP's method
...: timeit('new_to_coding(testrows)', setup, number=5)
Out[9]: 14.014577565016225
In [10]: # using `taras`
...: timeit('taras_memory_op(testrows)', setup, number=5)
Out[10]: 33.28037207596935
所以，我建议另一个解决方案。警告！以下解决方案就地修改内部列表：
In [14]: def sanitize(rows):
...: result = []
...: for row in rows:
...: tail = True
...: maxidx = len(row) - 1
...: for i, item in enumerate(reversed(row)):
...: if item is None:
...: if tail:
...: row.pop()
...: else:
...: row[maxidx - i] = ''
...: else:
...: tail = False
...: if row:
...: result.append(row)
...: return result
...:
In [15]: setup = "from __main__ import new_to_coding, taras_memory_op, sanitize, test_set; testrows = test_set()"
In [16]: # using `sanitize`
...: timeit('sanitize(testrows)', setup, number=5)
Out[16]: 8.261458976892754
In [17]: sanitize(test_set()) == new_to_coding(test_set())
Out[17]: True
所以，使用我最初制作的测试集：
In [18]: rows = [["row 1 index 0",None,"row 1 index 2",None,None],
...: [None,"row 2 index 1",None,None,None],
...: [None,None,None,None,None]]
In [19]:
In [19]: rows = [r*1000 for r in rows]
In [20]: rowsbig = [list(r) for _ in range(1000) for r in rows]
In [21]: rows = rowsbig
In [22]: del rowsbig
In [23]: def test_set(source=rows):
...: return [list(r) for r in source]
...:
In [24]: setup = "from __main__ import new_to_coding, taras_memory_op, sanitize, test_set; testrows = test_set()"
In [25]: # using `taras`
...: timeit('taras_memory_op(testrows)', setup, number=10)
Out[25]: 6.563127358909696
In [26]: # using OP's method
...: timeit('new_to_coding(testrows)', setup, number=10)
Out[26]: 10.173962660133839
In [27]: # using `sanitize`
...: timeit('sanitize(testrows)', setup, number=10)
Out[27]: 6.3629974271170795
答案 2 :(得分：1)
我确信有一种更紧凑的方式，但这是我对列表理解的看法：
data = [list(map(lambda x: '' if x is None else x, row)) for row in rows]
data = [row[:max(i for i, e in enumerate(row, 1) if e is not '')] for row in data if set(row) != {''}]