在Python中用元组(字符串)编写csv

时间:2016-05-13 17:06:55

标签: python regex string python-3.x csv

要了解我当前的问题,这里有一些关于更广泛问题的背景信息:

我有一个由多个文档组成的大文本文件。我需要找到一种方法将这个文件组织成它的组成部分。遗憾的是,所有单个文档都有不同的格式,其中唯一的共同点是每个文档的头部都包含日期,每次都以相同的格式编写:dd MONTH yyyy。我使用日期作为书挡来隔离它们之间的文本。

#the date pattern with positive lookbehind
bookend_1 = "(?<=\d{1,2}\sJANUARY\s\d{4}|\d{1,2}\sFEBRUARY\s\d{4}|\d{1,2}\sMARCH\s\d{4}|\d{1,2}\sAPRIL\s\d{4}|\d{1,2}\sMAY\s\d{4}|\d{1,2}\sJUNE\s\d{4}|\d{1,2}\sJULY\s\d{4}|\d{1,2}\sAUGUST\s\d{4}|\d{1,2}\sSEPTEMBER\s\d{4}|\d{1,2}\sOCTOBER\s\d{4}|\d{1,2}\sNOVEMBER\s\d{4}|\d{1,2}\sDECEMBER\s\d)"

#The date pattern with positive lookahead
bookend_2 = "(?=\d{1,2}\sJANUARY\s\d{4}|\d{1,2}\sFEBRUARY\s\d{4}|\d{1,2}\sMARCH\s\d{4}|\d{1,2}\sAPRIL\s\d{4}|\d{1,2}\sMAY\s\d{4}|\d{1,2}\sJUNE\s\d{4}|\d{1,2}\sJULY\s\d{4}|\d{1,2}\sAUGUST\s\d{4}|\d{1,2}\sSEPTEMBER\s\d{4}|\d{1,2}\sOCTOBER\s\d{4}|\d{1,2}\sNOVEMBER\s\d{4}|\d{1,2}\sDECEMBER\s\d)"

#using the bookends to find the text in between dates
docs = regex.findall(bookend_1+'(.*?)'+ bookend_2, psc_comm_raw, re.DOTALL|re.MULTILINE) 

使用正则表达式,我创建了两个列表:所有日期中的一个,日期之间发生的所有文本段落之一。我把这些列表压缩成一个元组。我无法将它们压缩成字典,因为有些日期会重复。

psc_comm_tuple = list(zip(date, docs))

以下是几行psc_comm_tuple。

[('27 JULY 2004',
  ' ADDIS ABABA, ETHIOPIA\n\nPSC/PR/Comm.(XIII)\n\nCOMMUNIQUÉ\n\nPSC/PR/Comm.(XIII) Page l\n\nCOMMUNIQUÉ OF THE THIRTEENTH MEETING OF THE PEACE AND SECURITY COUNCIL\n\nThe Peace and Security Council (PSC) of the African Union (AU), at its thirteenth meeting, held on 27 July 2004, adopted the following communiqué on the crisis in the Darfur region of the Sudan:\n\nCouncil,\n\n1.\tReiterates its deep concern over the grave situation that still prevails in the Darfur region of the Sudan, in particular the continued attacks by the Janjaweed militia against the civilian population, as well as other human rights abuses and the humanitarian crisis;\n\n2.\tUnderlines the urgent need to implement decision AU/Dec.54(111) on Darfur, adopted by the 3rd Ordinary Session of the Assembly...'),
 ('29 JANUARY 2001',
  '\n\nThe Central Organ of the OAU Mechanism for Conflict Prevention, Management and Resolution held its seventy-third * ordinary session at the level of Ambassadors on 29 January 2001, in Addis Ababa. The session was chaired by Ambassador Kati Ohara Korga, Permanent Representative of Togo to the OAU.\n\nHaving considered the Report of the Secretary General on the Democratic Republic of the Congo (DRC) and the situation in that country, the Central Organ:\n\n1.\tstrongly condemns the assassination of Pre...'),
 ('20 MARCH 2001',
  "\n\nThe Central Organ of the OAU Mechanism for Conflict Prevention, Management and Resolution held its 74th ordinary session at ambassadorial level, in Addis Ababa, Ethiopia, on Tuesday March 20, 2001. The session was chaired by Ambassador Ohara Korga, Permanent representative of Togo to the OAU....'),
 ('22 AUGUST 2001',
  '\n\nThe Central Organ of the OAU Mechanism for Conflict Prevention, Management and Resolution held its 75th Ordinary Session at Ambassadorial level in Addis Ababa, Ethiopia, on Wednesday 22 August 2001....')...]

我的最终目标是创建一个包含两列的CSV:一列用于日期,另一列用于与该日期关联的文本正文。

import csv
import os

with open('psc_comm.csv','w') as out:
    csv_out=csv.writer(out)
    csv_out.writerow(['date','text'])
    for row in psc_comm_tuple:
        csv_out.writerow(row)

当我将元组输出写入csv时,某些行完全正常。但是一些输出变得混乱 - 文本被分成看似随机的块,并且有空行,一排句子片段。这些事件有数百种。当我回顾原始文档并找到句子破坏的相应位置时,我无法看到文本本身的任何特殊或独特之处。没有特殊字符。它只是纯文本。但是,它们似乎确实是特别长的文本部分,因此我想知道CSV文件中的单个单元格可以包含的信息数量是否有限制。

我的问题是:为什么CSV输出在某些地方如此时髦而不是其他地方?每个单元格中可以放入多少文本?

Image of csv file

1 个答案:

答案 0 :(得分:1)

您没有提供足够的信息来识别问题,但Excel在读取带有嵌入换行符的CSV单元时往往会遇到问题,所以我的第一个猜测就是它出错的地方:你有一个嵌入换行符的csv, csvwriter可能以可逆的方式编写,但Excel无法正确解析。

换句话说,您的CSV文件可能没有任何问题;只是它被读入Excel的方式。你没有说你如何确定存在问题。

如果您的目标是生成Excel可以阅读的内容,我将免除CSV格式并直接获取电子表格。模块openpyxl可以生成xlsx文档,效果很好。