Question

我有数千个我将要处理的XML文件，它们具有相似的格式，但父母名称和父母的数量不同。通过书籍，谷歌，教程和只是尝试代码，我已经能够提取所有这些数据。例如，请参阅：Parsing xml to pandas data frame throws memory error和Dynamic search through xml attributes using lxml and xpath in python

然而，我意识到我提取的数据很差，每个家长都会重复一次“时间”。

这是我想要得到的。

Time   blah   abc
1200   100   2
1300   30    4
1400   70    2

以下是我知道如何获得的。但我目前的方法很笨重（我将在示例XML下面显示）

    child      Time   grandchild
0     blah     1200    100
1     blah     1300    30
...
n-2   abc      1200    2
n-1   abc      1300    4
n     abc      1400    2

示例XML格式

<outer>
   <inner>
      <parent name = "blah" id = "1"> 
         <child Time = "1200"> 
            <grandchild>100</grandchild>  
         </child>
         <child Time = "1300">
            <grandchild>30</grandchild>
         </child>
         <child Time = "1400">
            <grandchild>70</grandchild>
         </child>
      </parent>
      <parent name = "abc" id = "2"> 
         <child Time = "1200">   
            <grandchild>2</grandchild> 
         </child>
         <child Time = "1300">
            <grandchild>4</grandchild>
         </child>
         <child Time = "1400">
            <grandchild>2</grandchild>
         </child>
      </parent>      
      <parent name = "1234" id = "7734"> 
         <other> 12 </other>
      </parent> 
   </inner>
</outer>

以下是我如何获得输出：

from lxml import etree, objectify
from pandas import *
dTime=[]
dparent = []
dgrandchild=[]
for df in root.xpath('/*/*/*/parent/child'):
    dparent.append(df.getparent().attrib['name'])
    ## Iterate over attributes of time for specific parent
    for attrib in df.attrib:
    dTime.append(df.attrib[attrib])
        ## grandchild is a child of time, and iterate
        subfields = df.getchildren()
        for subfield in subfields:
         dgrandchild.append(subfield.text)
df=DataFrame({'Parent': dparent,'Time':dTime,'grandchild':dgrandchld})

我可以接受这个输出并重新塑造它，但这看起来效率低下且非常笨重。

我想我需要一些东西：

#this does not work
data = []
for elem in root.xpath('/*/*/*/parent/child'):
   elem_data = {}
   for attrib in elem.attrib:
       elem_data['Time'] = elem.attrib[attrib])
   for child in elem.getchildren():
       elem_data[getparent().attrib['name'])] = child.text
       data.append(elem_data)
ndata = DataFrame(data)

Answer 1

我建议首先解析一个DataFrame，类似于你已经如此（参见下面的实现），然后根据你的要求进行调整。

然后你正在寻找pivot：

In [11]: df
Out[11]:
  child  Time  grandchild
0  blah  1200         100
1  blah  1300          30
2   abc  1200           2
3   abc  1300           4
4   abc  1400           2

In [12]: df.pivot('Time', 'child', 'grandchild')
Out[12]:
child  abc  blah
Time
1200     2   100
1300     4    30
1400     2   NaN

我首先推荐parse from a file并将您想要的内容放入元组列表中：

from lxml import etree
root = etree.parse(file_name)

parents = root.getchildren()[0].getchildren()

In [21]: elems = [(p.attrib['name'], int(c.attrib['Time']), int(gc.text))
                      for p in parents
                      for c in p
                      for gc in c]

In [22]: elems
Out[22]:
[('blah', 1200, 100),
 ('blah', 1300, 30),
 ('blah', 1400, 70),
 ('abc', 1200, 2),
 ('abc', 1300, 4),
 ('abc', 1400, 2)]

对于多个文件，您可以在更长的列表理解中敲击它。 除非你有大量的xmls，否则不应该太慢（这里files是xmls的列表）......

elems = [(p.attrib['name'], int(c.attrib['Time']), int(gc.text))
            for f in files
            for p in etree.parse(f).getchildren()[0].getchildren()
            for c in p
            for gc in c]

将它们放在DataFrame中：

In [23]: pd.DataFrame(elems, columns=['child', 'Time', 'grandchild'])
Out[23]:
  child  Time grandchild
0  blah  1200        100
1  blah  1300         30
2  blah  1400         70
3   abc  1200          2
4   abc  1300          4
5   abc  1400          2

然后做转轴。：）

使用父属性作为列标题将XML提取到数据框中

1 个答案: