在Python中导入具有多个定界符的CSV文件

时间:2019-10-28 20:52:21

标签: python pandas csv dataframe delimiter

我正在尝试使用Python将数据文件导入笔记本。

这是实际数据: https://drive.google.com/file/d/1Fr5urzbuGx7QIg_2ueMXAAlDM9xU5e4P/view?usp=sharing

这是csv文件格式的一种方式:

"AwardNumber","Title","NSFOrganization","Program(s)","StartDate","LastAmendmentDate","PrincipalInvestigator","State","Organization","AwardInstrument","ProgramManager","EndDate","AwardedAmountToDate","Co-PIName(s)","PIEmailAddress","OrganizationStreet","OrganizationCity","OrganizationState","OrganizationZip","OrganizationPhone","NSFDirectorate","ProgramElementCode(s)","ProgramReferenceCode(s)","ARRAAmount","Abstract"
"1624943","Testing the Impact of Race on Jury Evaluations of Informants","SES","Sociology, Social Psychology, LSS-Law And Social Sciences","08/15/2016","07/17/2017","Mona Lynch","CA","University of California-Irvine","Standard Grant","Reggie Sheehan","06/30/2019","$353,747.00","","lynchm@uci.edu","141 Innovation Drive, Ste 250","Irvine","CA","926173213","9498247295","SBE","1331, 1332, 1372","9251","$0.00","An important body of legal scholarship has emerged about the justice risks associated with the use of informants, who provide information to law enforcement officials about criminal activity usually in exchange for leniency consideration or dismissal on a pending criminal charge. Despite the increasing concern, there has been very little empirical research on the use of informants as witnesses."
"1917573","States and Security: Border Orientation in the Modern World","SES","Political Science","08/15/2019","08/26/2019","Beth Simmons","PA","University of Pennsylvania","Standard Grant","Brian Humes","07/31/2021","$476,137.00","Michael Kenwick","simmons3@law.upenn.edu","Research Services","Philadelphia","PA","191046205","2158987293","SBE","1371","","$0.00","Border security is one of the most significant policy issues of our time. How do states benefit from globalization, while at the same time protecting a national space from unwanted influences, people, goods and activities?"
"1931871","CPS: Medium: A Secure, Trustworthy, and Reliable Air Quality Monitoring System for Smart and Connected Communities","SES","CPS-Cyber-Physical Systems","10/01/2019","10/24/2019","Haofei Yu","FL","University of Central Florida","Standard Grant","Sara Kiesler","09/30/2022","$1,198,111.00","Xinwen Fu, Deliang Fan, Haofei Yu, Kelly Stevens, Thomas Bryer","Haofei.Yu@ucf.edu","4000 CNTRL FLORIDA BLVD","Orlando","FL","328168005","4078230387","SBE","7918","7924, 9150","$0.00","A critical application of smart technologies is a smart, connected, and secured environmental monitoring network that can help administrators and researchers find better ways to incorporate evidence and data into public decision-making related to the environment."
"1922424","Standard Research: Consensus, Democracy, and the Public Understanding of Science","SES","STS-Sci, Tech & Society","09/01/2019","09/07/2019","James Weatherall","CA","University of California-Irvine","Continuing grant","Frederick Kronz","08/31/2022","$431,892.00","Cailin O'Connor","weatherj@uci.edu","141 Innovation Drive, Ste 250","Irvine","CA","926173213","9498247295","SBE","7603","1353","$0.00","This award supports a research project that studies how changing social networks influence public belief about science; it will focus specifically on how false beliefs can persist and spread even in evidence-rich environments, and how these beliefs in turn feed back into collective decision-making through democratic institutions."

我遇到的问题是,值不是仅由列分隔,而是也用引号引起来,这是必须的,因为其中一列包含大量的字符串文本。

这是正常导入的方式,但是出现错误。

import pandas as pd
import numpy as np

award = pd.read_csv('ses_awards.csv')
award.head()

预先感谢您的帮助!

2 个答案:

答案 0 :(得分:3)

您需要将quotechar='"'参数与pd.read_csv()函数一起使用,如下所示:

import pandas as pd
import numpy as np

award = pd.read_csv('ses_awards.csv', quotechar='"')
award.head()

Pandas documentation关于read_csv()

  

quotechar :str(长度1),可选
  用来表示   引用项目的开始和结束。引用的项目可以包含定界符   它将被忽略。

答案 1 :(得分:2)

我尝试了您提供的文件,但实际上给了我一个编码错误。

尝试以下编码:

pd.read_csv('ses_awards.csv', encoding = 'ISO-8859-1')