我一直在使用pandas对CSV文件进行一些有趣的过滤,但遇到了障碍。我试图检查我的索引列的乱码文本(非整数)数据,并删除这些行。我在尝试使用条件导入时尝试从数据框中删除它们,并且我尝试在之后迭代它们但没有成功。这是一个例子:
df = pd.read_csv(file, encoding='cp1252').set_index("numbers")
results = df[df["columnA"].str.contains("search_data") & ~df["columnB"].isin(seach_list)]
#I need to add to the above statement to check column "numbers" which I have set to be the index,
#to catch some expected garbled text and filter it out... because it is
#an integer, I can't use str.contains or isdigit or isalnum, I've tried to do len(df["columns"] < 20 , df.index < 20 .... i've tried
#i've tried a few other options at this point as well
# after bringing it in, I've also tried iterating through it:
#
for index, row in results.iterrows():
if not (isinstance( row["numbers"], int )):
print(str(row["numbers"]))
#append whole row to new dataframe
#This also didn't work
关于我能做什么的任何想法?
Example data in the "numbers columns = 329381432
Example garbled text in "numbers" column that I am
trying to keep from importing: äu$ÒÔ”5$ò"Â$”äu$ÒÔ”5$ò
作为旁注,我必须更改pd函数的编码,这样当有一些非utf-8数据时我仍然可以读取文件中的所有好数据...否则会抛出错误导入。
答案 0 :(得分:0)
您可以使用pd.to_numeric
将numbers
列转换为数字。所有非数字条目都将强制转换为NaN
,然后您可以删除这些行。
df = pd.read_csv(file, encoding='cp1252')
df['numbers'] = pd.to_numeric(df['numbers'], errors='coerce')
df = df.dropna(subset=['numbers']).set_index('numbers')