我有一个大文件，其中包含大量数据。我需要每5000行左右提取3行。数据文件的格式如下：
...
O_sh 9215 1.000000 -2.304400
-1.0680E+00 1.3617E+00 -5.7138E+00
O_sh 9216 1.000000 -2.304400
-8.1186E-01 -1.7454E+00 -5.8169E+00
timestep 501 9216 0 3 0.000500
20.54 -11.85 35.64
0.6224E-02 23.71 35.64
-20.54 -11.86 35.64
Li 1 6.941000 0.843200
3.7609E-02 1.1179E-01 4.1032E+00
Li 2 6.941000 0.843200
6.6451E-02 -1.3648E-01 1.0918E+01
...
我需要的是以“timestep”开头的行之后的三行，所以在这种情况下我需要3x3数组：
20.54 -11.85 35.64
0.6224E-02 23.71 35.64
-20.54 -11.86 35.64
。
然后我只需要一个数组中所有这些数组的平均值。只有一个数组，包含整个文件的每个数组中相同位置的每个元素的平均值。 我已经研究了一段时间，但我还没能正确提取数据。
非常感谢，这不是作业。你的建议将有助于科学的进步！ =）
谢谢，
答案 0 :(得分：3)
假设这不是作业，我认为正则表达式对于这个问题来说太过分了。如果您知道在“timetep”开始后需要三行，为什么不以这种方式解决问题：
Matrices = []
with open('data.txt') as fh:
for line in fh:
# If we see timestep put the next three lines in our Matrices list.
if line.startswith('timestep'):
Matrices.append([next(fh) for _ in range(3)])
根据评论 - 在这种情况下你使用next（fh）来保持文件句柄同步，当你想从中提取下三行时。谢谢！
答案 1 :(得分：2)
我建议使用coroutine（基本上是一个可以接受值的生成器，如果您不熟悉的话），以便在迭代文件时保持运行平均值。
def running_avg():
count, sum = 0, 0
value = yield None
while True:
if value:
sum += value
count += 1
value = yield(sum/count)
# array for keeping running average
array = [[running_avg() for y in range(3)] for x in range(3)]
# advance to first yield before we begin
[[elem.next() for elem in row] for row in array]
with open('data.txt') as f:
idx = None
for line in f:
if idx is not None and idx < 3:
for i, elem in enumerate(line.strip().split()):
array[idx][i].send(float(elem))
idx += 1
if line.startswith('timestep'):
idx = 0
要将转换array转换为平均值列表，只需调用每个协程next方法，它将返回当前平均值：
averages = [[elem.next() for elem in row] for row in array]
你会得到类似的东西：
averages = [[20.54, -11.85, 35.64], [0.006224, 23.71, 35.64], [-20.54, -11.86, 35.64]]
答案 2 :(得分：1)
好的，你可以这样做：
算法：
Read the file line by line
if the line starts with "timestep":
read the next three lines
take the average as needed
代码：
def getArrays(f):
answer = [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
count = 0
line = f.readline()
while line:
if line.strip().startswith("timestep"):
one, two, three = getFloats(f.readline().strip()), getFloats(f.readline().strip()), getFloats(f.readline().strip())
answer[0][0] = ((answer[0][0]*count) + one[0])/(count+1)
answer[0][1] = ((answer[0][0]*count) + one[1])/(count+1)
answer[0][2] = ((answer[0][0]*count) + one[2])/(count+1)
answer[1][0] = ((answer[0][0]*count) + two[0])/(count+1)
answer[1][1] = ((answer[0][0]*count) + two[1])/(count+1)
answer[1][2] = ((answer[0][0]*count) + two[2])/(count+1)
answer[2][0] = ((answer[0][0]*count) + three[0])/(count+1)
answer[2][1] = ((answer[0][0]*count) + three[1])/(count+1)
answer[2][2] = ((answer[0][0]*count) + three[2])/(count+1)
line = f.readline()
count += 1
return answer
def getFloats(line):
answer = []
for num in line.split():
if "E" in num:
parts = num.split("E")
base = float(parts[0])
exp = int(parts[1])
answer.append(base**exp)
else:
answer.append(float(num))
return answer
answer现在是所有3x3阵列的列表。我不知道你想如何进行平均，所以如果你发布它，我可以将它合并到这个算法中。否则，你可以编写一个函数来获取我的数组并计算平均值。
希望这有帮助
答案 3 :(得分：0)
在inspectorG4dget和g.d.d.c的帖子的基础上，这里有一个应该进行读取，解析和平均的版本。请指出我的错误！ ：）
def averageArrays(filename):
# initialize average variables then,
# open the file and iterate through the lines until ...
answer, count = [[0.0]*3 for _ in range(3)], 0
with open(filename) as fh:
for line in fh:
if line.startswith('timestep'): # ... we find 'timestep'!
# so , we read the three lines and sanitize them
# conversion to float happens here, which may be slow
raw_mat = [fh.next().strip().split() for _ in range(3)]
mat = []
for row in raw_mat:
mat.append([float(item) for item in row])
# now, update the running average, noting overflows as by
# http://invisibleblocks.wordpress.com/2008/07/30/long-running-averages-without-the-sum-of-preceding-values/
# there are surely more pythonic ways to do this
count += 1
for r in range(3):
for c in range(3):
answer[r][c] += (mat[r][c] - answer[r][c]) / count
return answer
答案 4 :(得分：0)
import re
from itertools import imap
text = '''O_sh 9215 1.000000 -2.304400
-1.0680E+00 1.3617E+00 -5.7138E+00
O_sh 9216 1.000000 -2.304400
-8.1186E-01 -1.7454E+00 -5.8169E+00
timestep 501 9216 0 3 0.000500
20.54 -11.85 35.64
0.6224E-02 23.71 35.64
-20.54 -11.86 35.64
Li 1 6.941000 0.843200
3.7609E-02 1.1179E-01 4.1032E+00
Li 2 6.941000 0.843200
6.6451E-02 -1.3648E-01 1.0918E+01
O_sh 9215 1.000000 -2.304400
-1.0680E+00 1.3617E+00 -5.7138E+00
O_sh 9216 1.000000 -2.304400
-8.1186E-01 -1.7454E+00 -5.8169E+00
timestep 501 9216 0 3 0.000500
80.80 -14580 42.28
7.5224E-01 777.1 42.28
140.54 -33.86 42.28
Li 1 6.941000 0.843200
3.7609E-02 1.1179E-01 4.1032E+00
Li 2 6.941000 0.843200
6.6451E-02 -1.3648E-01 1.0918E+01'''
lin = '\r?\n{0}*({1}+){0}+({1}+){0}+({1}+){0}*'
pat = ('^timestep.+'+3*lin).format('[ \t]','[.\deE+-]')
regx = re.compile(pat,re.MULTILINE)
def moy(x):
return sum(map(float,x))/len(x)
li = map(moy,zip(*regx.findall(text)))
n = len(li)
g = iter(li).next
res = [(g(),g(),g()) for i in xrange(n//3)]
print res
结果
[(50.67, -7295.925, 38.96), (0.379232, 400.40500000000003, 38.96), (60.0, -22.86, 38.96)]