我创建了以下函数,该函数将XML文件转换为DataFrame。此功能适用于小于1 GB的文件,以及大于RAM(13GB Google Colab RAM)崩溃的文件。如果我在Jupyter Notebook(4GB Laptop RAM)上本地尝试,也会发生同样的情况。有没有一种方法可以优化代码?
代码
#Libraries
import pandas as pd
import xml.etree.cElementTree as ET
#Function to convert XML file to Pandas Dataframe
def xml2df(file_path):
#Parsing XML File and obtaining root
tree = ET.parse(file_path)
root = tree.getroot()
dict_list = []
for _, elem in ET.iterparse(file_path, events=("end",)):
if elem.tag == "row":
dict_list.append(elem.attrib) # PARSE ALL ATTRIBUTES
elem.clear()
df = pd.DataFrame(dict_list)
return df
XML文件的一部分(“ Badges.xml”)
<badges>
<row Id="82946" UserId="3718" Name="Teacher" Date="2008-09-15T08:55:03.923" Class="3" TagBased="False" />
<row Id="82947" UserId="994" Name="Teacher" Date="2008-09-15T08:55:03.957" Class="3" TagBased="False" />
<row Id="82949" UserId="3893" Name="Teacher" Date="2008-09-15T08:55:03.957" Class="3" TagBased="False" />
<row Id="82950" UserId="4591" Name="Teacher" Date="2008-09-15T08:55:03.957" Class="3" TagBased="False" />
<row Id="82951" UserId="5196" Name="Teacher" Date="2008-09-15T08:55:03.957" Class="3" TagBased="False" />
<row Id="82952" UserId="2635" Name="Teacher" Date="2008-09-15T08:55:03.957" Class="3" TagBased="False" />
<row Id="82953" UserId="1113" Name="Teacher" Date="2008-09-15T08:55:03.957" Class="3" TagBased="False" />