我正在编写一个应该删除重复条目的脚本。数据中的某些人输入了他们的名字两次,因为他们有两个电话号码,而且由于电话号码字段不是一个数组,要输入多个,他们输入了多个条目。
我的脚本使用与列名对应的键将条目更改为字典,然后遍历每一行。有一个循环的主循环遍历每一行,然后是一个嵌套的for循环,它遍历每个元素的所有元素,比较它们以检测重复。当我点击重复时,我的代码应该比较手机,电子邮件和网站,然后将它们附加到区域,如果它们是唯一的/不匹配的。
这是我的代码:
import csv
# This function takes a tab-delim csv and merges the ones with the same name but different phone / email / websites.
def merge_duplicates(sheet):
myjson = [] # myjson = list of dictionaries where each dictionary
with(open("ieca_first_col_fake_text.txt", "rU")) as f:
sheet = csv.DictReader(f,delimiter="\t")
for row in sheet:
myjson.append(row)
write_file = csv.DictWriter(open('duplicates_deleted.csv','w'), ['name','phone','email','website'], restval='', delimiter = '\t')
for row in myjson:
# convert phone, email, and web to lists so that extra can be appended
row['phone'] = row['phone'].split()
row['email'] = row['email'].split()
row['website'] = row['website'].split()
print row
for i in len(myjson):
# if the names match, check to see if phone, em, web match. If any match, append to first row.
try:
if myjson[i]['name'] == myjson[i+1]['name']:
if myjson[i]['phone'] != myjson[i+1]['phone']:
myjson[i]['phone'].append(myjson[i+1]['phone'])
# if row['email'] != myjson[rowvalue+1]['email']:
# row['email'].append(myjson[rowvalue+1]['email'])
# if row['website'] != myjson[rowvalue+1]['website']:
# row['website'].append(myjson[rowvalue+1]['website'])
except IndexError:
print("We're at the end now")
write_file.writerow(row)
merge_duplicates('ieca_first_col_fake_text.txt')
所以一切都在我的代码中花哨,然后它击中了第一个副本,我得到了这个错误:
{'website': [], 'phone': [], 'name': 'Diane Grant Albrecht M.S.', 'email': []}
{'website': ['www.got.com'], 'phone': ['111-222-3333'], 'name': 'Lannister G. Cersei M.A.T., CEP', 'email': ['cersei@got.com']}
{'website': [], 'phone': [], 'name': 'Argle D. Bargle Ed.M.', 'email': []}
{'website': ['www.daManWithThePlan.com'], 'phone': ['000-000-1111'], 'name': 'Sam D. Man Ed.M.', 'email': ['dman123@gmail.com']}
Traceback (most recent call last):
File "/Users/samuelfinegold/Documents/noodle/delete_duplicates.py", line 40, in <module>
merge_duplicates('ieca_first_col_fake_text.txt')
File "/Users/samuelfinegold/Documents/noodle/delete_duplicates.py", line 20, in merge_duplicates
row['email'] = row['email'].split()
AttributeError: 'NoneType' object has no attribute 'split'
logout
非常感谢你的帮助!
Ex数据是否有帮助:
name phone email website
Diane Grant Albrecht M.S.
"Lannister G. Cersei M.A.T., CEP" 111-222-3333 cersei@got.com www.got.com
Argle D. Bargle Ed.M.
Sam D. Man Ed.M. 000-000-1111 dman123@gmail.com www.daManWithThePlan.com
Sam D. Man Ed.M.
Sam D. Man Ed.M. 111-222-333 dman123@gmail.com www.daManWithThePlan.com
D G Bamf M.S.
Amy Tramy Lamy Ph.D.
答案 0 :(得分:3)
错误是,如果row['phone']
为None
,则无法将其拆分。
你可以这样做
row['phone'] = row['phone'].split() if row['phone'] else []
row['email'] = row['email'].split() if row['email'] else []
row['website'] = row['website'].split() if row['website'] else []
[]
可以替换为您要指定的任何默认值(例如:None
或""
)。
更清洁的方式是
row['phone'] = row['phone'].split() if row.get('phone') else []
row['email'] = row['email'].split() if row.get('email') else []
row['website'] = row['website'].split() if row.get('website') else []
答案 1 :(得分:1)
就个人而言,我会使用and
来执行此操作:
row['email'] = row.get('email',[]) and row['email'].split()
逻辑与:
相同if row.get('email'):
row['email'] = row['email'].split()
虽然严格来说,如果密钥丢失(或电子邮件已经写入列表),这会重新分配,所以您可能希望这样做:
# you could also use hasattr(row['email'],'split')
if 'email' in row and isinstance(row['email'],str):
row['email'] = row['email'].split()