拆分生成器，独立处理并再次压缩

Question

我该怎么做？

sample_generator = get_sample_generator() # This generator yields tuples (label(i), data(i))


label_generator = ???
data_generator = ???

processed_data_generator = process_data_generator(data_generator)

processed_sample_generator = zip(label_generator, processed_data_generator) # Labels and corresponding data again as tupel (label(i), processed_data(i))

显然我不能将sample_generator map（）映射到标签和数据生成器，因为两者都会向前移动sample_generator ......

Answer 1

使用tee复制原始生成器，然后拆分其中一个副本以获取数据。

sample1, sample2 = itertools.tee(sample_generator, 2)
processed_data_generator = process_data_generator(d for (l,d) in sample1)
processed_sample_generator = sample2

Answer 2

使用itertools.tee将生成器拆分为2.您仍然需要单个进程来拆分结果值。

from itertools import count

def generator():
    for c in count():
        yield 'label {}'.format(c), 'data {}'.format(c)

from itertools import tee

label_generator, data_generator = tee(generator())

print("labels:")
for _ in range(5):
    print(next(label_generator)[0])
print()
print("data:")
for _ in range(5):
    print(next(data_generator)[1])

Answer 3

展开sample_generator并同时取消label_generator和data_generator：

import itertools
label_generator = data_generator = itertools.chain.from_iterable(sample_generator)