Question

我有一些看起来像这样的数据：

description "export"
source "factory1"
source "factory2"
source "factory3"
destination "customer1"
destination "customer2"
shipdate "asap"

description "export"
source "factory4"
source "factory5"
source "factory6"
destination "customer1"
shipdate "30"

我现在正在尝试创建一个看起来像这样的csv文件：

description,source,destination,shipdate
export,factory1,customer1,asap
export,factory2,customer1,asap
export,factory3,customer1,asap
export,factory1,customer2,asap
export,factory2,customer2,asap
export,factory3,customer2,asap
export,factory1,customer1,asap
export,factory2,customer1,asap
export,factory3,customer1,asap
export,factory4,customer1,30
export,factory5,customer1,30
export,factory6,customer1,30

数据块以python列表的形式发送给我，因此我目前正在对其进行循环浏览，然后基于将它们放入其他列表的第一个单词。但是，可能有很多入门方法。

到目前为止，我的代码看起来像，但是您可以看到它无法解决我的问题：

sourcelist = []
destlist = []
for item in list:
  if "source" in item:
    sourcelist.append(item)
  if "destination" in item:
    destlist.append(item)

感谢您的帮助！即使这意味着我需要重写代码！

Answer 1

由于每行本身都不足够，因此必须累积数据。根据您的描述和示例，您可以按块进行此操作。只需累积一个块的 all 个字段-唯一的字段将只有一个项目。

您可以使用生成器高效地解析块：

def parse_blocks(source: 'Iterable[str]'):
    block = {}
    for line in source:
        if not line:  # delimiter between blocks
            yield block
            block = {}
        else:
            key, value = line.split()
            block.setdefault(key, []).append(value.strip('"'))
    if block:
        yield block

这为您提供了一个可迭代的块，例如

{'description': ['export'], 'source': ['factory1', 'factory2', 'factory3'], 'destination': ['customer1', 'customer2'], 'shipdate': ['asap']}, ...

对于每个块，都需要跨字段的所有组合。 itertools.product开箱即用。

import itertools

def merge_lines(blocks: 'Dict[str, List[str]]', *fields: 'str'):
    for block in blocks:
        yield from itertools.product(
            *(block[key] for key in fields)
        )

这提供了每行的数据作为元组的可迭代项：

('export', 'factory1', 'customer1', 'asap'), ('export', 'factory1', 'customer2', 'asap'), ...

您可以将其直接输入csv，或以您认为合适的任何方式对其进行处理。

import csv
import sys

fields = 'description', 'source', 'destination', 'shipdate'

writer = csv.writer(sys.stdout)  # or write to a file, pipe, ...
writer.writerow(fields)
for data in merge_lines(parse_blocks(input_list), *fields):  # insert your input here
    writer.writerow(data)

这将产生所需的csv输出：

description,source,destination,shipdate
export,factory1,customer1,asap
export,factory1,customer2,asap
export,factory2,customer1,asap
export,factory2,customer2,asap
export,factory3,customer1,asap
export,factory3,customer2,asap
export,factory4,customer1,30
export,factory5,customer1,30
export,factory6,customer1,30

Answer 2

您在这里：

import pandas as pd
import re
import itertools

# setup test data
raw_data_1 = ['description export', 'source factory 1', 'source factory 2', 'source factory 3', 'destination customer 1',
  'destination customer 2', 'shipdate asap']

raw_data_2 = ['description export', 'source factory 4', 'source factory 5', 'source factory 6', 'destination customer 1',
   'shipdate 30']

# create list of input data
data_list = [raw_data_1, raw_data_2]


# collect data from string
collected_data = []
for item in data_list:
    description = 0
    source_data = []
    destination_data = []
    ship_date = 0
    for data in item:
        if 'description' in data:
            description = re.sub('description ', '', data)
        elif 'source' in data:
            source = re.sub('source ', '', data)
            source_data.append(source)
        elif 'destination' in data:
            destination = re.sub('destination ', '', data)
            destination_data.append(destination)
        elif 'shipdate' in data:
            ship_date = re.sub('shipdate ', '', data)

    # create combinations
    combination_data = list(itertools.product(source_data, destination_data))

    # extend combinations data
    for item in combination_data:
        out = [description] + list(item) + [ship_date]
        collected_data.append(out)


# past data into data frame
data = pd.DataFrame(collected_data, columns=['description', 'source', 'destination','shipdate'])

# save data to file
data.to_csv('data.csv', index=False)

输出：

  description     source destination shipdate
0      export  factory 1  customer 1     asap
1      export  factory 1  customer 2     asap
2      export  factory 2  customer 1     asap
3      export  factory 2  customer 2     asap
4      export  factory 3  customer 1     asap
5      export  factory 3  customer 2     asap
6      export  factory 4  customer 1       30
7      export  factory 5  customer 1       30
8      export  factory 6  customer 1       30

遍历列表，在csv中多次重用某些值

2 个答案: