Question

我正在编写一个脚本来处理一些csv数据，以便在d3.js地图上呈现它，我在这句话之后立即提供完整的脚本，但仅针对上下文，我的问题的重要部分如下。

# The purpose of this script is the refinement of the job data attained from the
# JSI as it is rendered by the `csv generator` contributed by Luis for purposes
# of presentation on the dashboard map. 

import csv

# The number of columns
num_headers = 9

# Remove invalid characters from records
def url_escaper(data):
  for line in data:
    yield line.replace('&','&amp;')

# Be sure to configure input & output files
with open("input.txt", 'r') as file_in, open("try_this_output.txt", 'w') as file_out:
    csv_in = csv.reader( url_escaper( file_in ) )
    csv_out = csv.writer(file_out)

    # Get rid of rows that have the wrong number of columns
    # and rows that have only whitespace for a columnar value
    for i, row in enumerate(csv_in, start=1):
        for e in row:
            if "|" in e:
                e = e.split(";")[0]
        if not [e for e in row if not e.strip()]:
            if len(row) == num_headers:
                csv_out.writerow(row)
        else:
            print "line %d is malformed" % i

有些柱状值的结构如下：

linux|devops|firewall|vmware|.net-framework|.net|paas

我想使用以下代码段将它们分割：

e.split("|")[0]

这样我只能在＆＃34; |＆＃34;之前留下文字的第一部分，即在上面的示例linux中。

我需要将此处理过的数据写入输出文件。

我知道该代码段有效，但我无法弄清楚如何将其与我的管道相匹配。

这是我关注的部分：

for i, row in enumerate(csv_in, start=1):
    for e in row:
        if "|" in e:
            e = e.split("|")[0]
    if not [e for e in row if not e.strip()]:
        if len(row) == num_headers:
            csv_out.writerow(row)
    else:
        print "line %d is malformed" % i

特别是：

    for e in row:
        if "|" in e:
            e = e.split(";")[0]

很明显，这不是实现这一目标的方法。

输入数据的一个例子是：

http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJpIjoiMjk1MzYyMDY2IiwicyI6ImhiTUN6MTdUUkVPdWl5NUI2bDdwQXcifQ.A6MlT_WKpLx763hZe44X4pQ0KOMHYuITosCIwuMbPxM,Technical Account Manager, Technical Delivery Manager - Cloud,Peopleworks,Farnborough,51.293999,-0.754624,United Kingdom,linux|devops|firewall|vmware|.net-framework|.net|paas,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJpIjoiMzA5MzE5OTExIiwicyI6Ik9feVBUT1VNVC0tcUZ2N1FvRWNVU1EifQ.C8ZAc9RAFSMdyaCaIIMB51-jGS01Az29VY8Dblc7QM4,Management Consultant - Utilities Smart Energy,Capgemini Consulting,Lee,51.451818,-0.02806,United Kingdom,leadership|database|project management|design|scada,1

，理想的输出是

http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJpIjoiMjk1MzYyMDY2IiwicyI6ImhiTUN6MTdUUkVPdWl5NUI2bDdwQXcifQ.A6MlT_WKpLx763hZe44X4pQ0KOMHYuITosCIwuMbPxM,Technical Account Manager, Technical Delivery Manager - Cloud,Peopleworks,Farnborough,51.293999,-0.754624,United Kingdom,linux,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJpIjoiMzA5MzE5OTExIiwicyI6Ik9feVBUT1VNVC0tcUZ2N1FvRWNVU1EifQ.C8ZAc9RAFSMdyaCaIIMB51-jGS01Az29VY8Dblc7QM4,Management Consultant - Utilities Smart Energy,Capgemini Consulting,Lee,51.451818,-0.02806,United Kingdom,leadership,1

Answer 1

您可以使用正则表达式解决问题。

我抓住您的输入数据并将其放入文件＆quot; input.txt＆＃39;

http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJpIjoiMjk1MzYyMDY2IiwicyI6ImhiTUN6MTdUUkVPdWl5NUI2bDdwQXcifQ.A6MlT_WKpLx763hZe44X4pQ0KOMHYuITosCIwuMbPxM,Technical Account Manager, Technical Delivery Manager - Cloud,Peopleworks,Farnborough,51.293999,-0.754624,United Kingdom,linux|devops|firewall|vmware|.net-framework|.net|paas,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJpIjoiMzA5MzE5OTExIiwicyI6Ik9feVBUT1VNVC0tcUZ2N1FvRWNVU1EifQ.C8ZAc9RAFSMdyaCaIIMB51-jGS01Az29VY8Dblc7QM4,Management Consultant - Utilities Smart Energy,Capgemini Consulting,Lee,51.451818,-0.02806,United Kingdom,leadership|database|project management|design|scada,1

您的预期结果：

http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJpIjoiMjk1MzYyMDY2IiwicyI6ImhiTUN6MTdUUkVPdWl5NUI2bDdwQXcifQ.A6MlT_WKpLx763hZe44X4pQ0KOMHYuITosCIwuMbPxM,Technical Account Manager, Technical Delivery Manager - Cloud,Peopleworks,Farnborough,51.293999,-0.754624,United Kingdom,linux,1
http://www.edsa-project.eu/adzuna/eyJhbGciOiJIUzI1NiJ9.eyJpIjoiMzA5MzE5OTExIiwicyI6Ik9feVBUT1VNVC0tcUZ2N1FvRWNVU1EifQ.C8ZAc9RAFSMdyaCaIIMB51-jGS01Az29VY8Dblc7QM4,Management Consultant - Utilities Smart Energy,Capgemini Consulting,Lee,51.451818,-0.02806,United Kingdom,leadership,1

这是我使用的代码：

import re

src = open('input.txt')
output = open('output.txt', 'w')


pat = r'([^,]*\|[^,]*)'

for line in src:
    the_search = re.search(pat, line) # Search the line for something containing '|'
    if the_search:
        the_group = the_search.group(0) # Grab the capture group
        value = the_group.split("|")[0] # Grab the first item after splitting based on '|'
        new_line = re.sub(pat, value, line) # Use re.sub to replace that entire pattern with the value
        output.write(new_line)

src.close()
output.close()

您需要定制此解决方案以适应您尝试做的事情，但正则表达式应该有效。

在csv

1 个答案: