使用python解析xml(查找包含特定文本的标签)

时间:2019-02-07 12:35:58

标签: xml python-3.x csv lxml export-to-csv

我的任务是处理xml文件,以查找特定元素并将其导出到csv文件中。

我对于在相同标签中保留的某些信息特别有麻烦:

<name>text</name>
<value>value</value>

每个名称标签包含一个不同的值,我只需要其中一些。 我尝试使用以下代码遍历文件:

try:
        descr = member.find('.//name').text
        if descr == 'description':
            plugin.append(descr)
    except AttributeError:
        descr = 'Unknown'
        plugin.append(descr)

但它只会返回“未知”

我的整个代码就是这样(未完成):

import xml.etree.ElementTree as ET
import csv

tree = ET.parse('plugins.xml')
root = tree.getroot()

nessus_out = open('/home/rj/Documents/python/nessus_out.csv', 'w')

csvwriter = csv.writer(nessus_out)

for member in root.findall('nasl'):
    plugin = []

    id = member.find('script_id').text
    plugin.append(id)

    name = member.find('script_name').text
    plugin.append(name)

    family = member.find('script_family').text
    plugin.append(family)

    #for each in member.iterfind('nasl'):
    try:
        solution = member.xpath('.//name/text()')
        if solution == 'solution':
            plugin.append(solution)
    except AttributeError:
        solution = 'Unknown'
        plugin.append(solution)
    csvwriter.writerow(plugin)
nessus_out.close()

最终目标是搜索“解决方案”,并从其标签中获取相应的值。

xml结构如下:

nasl_plugins
nasl_plugins/nasl
nasl_plugins/nasl/filename
nasl_plugins/nasl/script_id
nasl_plugins/nasl/script_name
nasl_plugins/nasl/script_family
nasl_plugins/nasl/attributes/attribute/name
nasl_plugins/nasl/attributes/attribute/value

对于丹尼尔:

Xml代码段:

<nasl>
<filename>fedora_2017-c3149b5fcb.nasl</filename>
<script_id>101028</script_id>
<script_name>Fedora 25 : xen (2017-c3149b5fcb)</script_name>
<script_version>$Revision: 1.5 $</script_version>
<script_copyright>This script is Copyright (C) 2017-2018 Tenable Network Security, Inc.</script_copyright>
<script_family>Fedora Local Security Checks</script_family>
<cves>
 <cve>CVE-2017-10911</cve>
 <cve>CVE-2017-10912</cve>
 <cve>CVE-2017-10913</cve>
 <cve>CVE-2017-10915</cve>
 <cve>CVE-2017-10916</cve>
 <cve>CVE-2017-10917</cve>
 <cve>CVE-2017-10918</cve>
 <cve>CVE-2017-10919</cve>
 <cve>CVE-2017-10920</cve>
 <cve>CVE-2017-10923</cve>
</cves>
<bids>
</bids>
<xrefs>
 <xref>FEDORA:2017-c3149b5fcb</xref>
 <xref>IAVB:2017-B-0074</xref>
</xrefs>
<dependencies>
 <dependency>ssh_get_info.nasl</dependency>
</dependencies>
<required_keys>
 <key>Host/local_checks_enabled</key>
 <key>Host/RedHat/release</key>
 <key>Host/RedHat/rpm-list</key>
</required_keys>
<attribute> 
  <name>plugin_type</name> 
  <value>local</value> 
</attribute> 
<attribute> 
  <name>plugin_modification_date</name> 
  <value>2018/02/02</value> 
</attribute> 
<attribute> 
  <name>stig_severity</name> 
  <value>I</value> 
</attribute> 
<attribute> 
  <name>cvss_base_score</name> 
  <value>10.0</value> 
</attribute> 
</attributes> 

我正在寻找的是stig_severity,base_cvss_score和som其他值。.因此,我的推理是搜索,然后它们向下移动一行并获得该值。.至于csv,我需要将它合二为一生产线插件,因此采用以下格式: id,name,family,solution,description,synopsis,base_cvss_score,plugin_type,stig_severity,然后在下一行显示下一个插件的值。

1 个答案:

答案 0 :(得分:0)

您想要的某些值似乎是nasl的直接子级,而某些值在attributes/attribute中。

可以可以做的是有两个列表(或元组);一个带有确切的元素名称,另一个带有确切的属性名称(attribute/name)。

注意:这听起来可能有些混乱,因为在这种情况下,“属性名称”实际上是一个名为“属性”的元素,而其子元素名为“ name”,而不是一个真正的XML属性,名为“ name”。

组合这些元组将为您提供CSV中的所有字段。您可以使用它来构建一个包含所有默认值为Unknown的字段的字典。

然后,您可以遍历两个元组以构建两种不同类型的XPath。如果元素存在,则在字典中更新文本值。否则,该值仍为Unknown

示例...

XML输入(test.xml)

<nasl_plugins>
    <nasl>
        <filename>fedora_2017-c3149b5fcb.nasl</filename>
        <script_id>101028</script_id>
        <script_name>Fedora 25 : xen (2017-c3149b5fcb)</script_name>
        <script_version>$Revision: 1.5 $</script_version>
        <script_copyright>This script is Copyright (C) 2017-2018 Tenable Network Security,
            Inc.</script_copyright>
        <script_family>Fedora Local Security Checks</script_family>
        <cves>
            <cve>CVE-2017-10911</cve>
            <cve>CVE-2017-10912</cve>
            <cve>CVE-2017-10913</cve>
            <cve>CVE-2017-10915</cve>
            <cve>CVE-2017-10916</cve>
            <cve>CVE-2017-10917</cve>
            <cve>CVE-2017-10918</cve>
            <cve>CVE-2017-10919</cve>
            <cve>CVE-2017-10920</cve>
            <cve>CVE-2017-10923</cve>
        </cves>
        <bids> </bids>
        <xrefs>
            <xref>FEDORA:2017-c3149b5fcb</xref>
            <xref>IAVB:2017-B-0074</xref>
        </xrefs>
        <dependencies>
            <dependency>ssh_get_info.nasl</dependency>
        </dependencies>
        <required_keys>
            <key>Host/local_checks_enabled</key>
            <key>Host/RedHat/release</key>
            <key>Host/RedHat/rpm-list</key>
        </required_keys>
        <attributes>
            <attribute>
                <name>plugin_type</name>
                <value>local</value>
            </attribute>
            <attribute>
                <name>plugin_modification_date</name>
                <value>2018/02/02</value>
            </attribute>
            <attribute>
                <name>stig_severity</name>
                <value>I</value>
            </attribute>
            <attribute>
                <name>cvss_base_score</name>
                <value>10.0</value>
            </attribute>
        </attributes>
    </nasl>
</nasl_plugins>

Python 3.x

import csv
from lxml import etree

elem_names = ('script_id', 'script_name', 'script_family')
attr_names = ('solution', 'description', 'synopsis', 'cvss_base_score', 'plugin_type',
              'stig_severity')
field_names = elem_names + attr_names

with open('test.csv', 'w', newline='', encoding='utf8') as xml_data_to_csv:

    csv_writer = csv.DictWriter(xml_data_to_csv, fieldnames=field_names, 
                                quoting=csv.QUOTE_ALL)

    csv_writer.writeheader()

    tree = etree.parse('test.xml')

    for nasl in tree.xpath('.//nasl'):
        # Build a dict containing all of the "field_names" with default values of "Unknown".
        values = {key: 'Unknown' for key in field_names}

        # Process the direct children of "nasl".
        for elem_name in elem_names:
            for child in nasl.xpath(f'*[self::{elem_name}]'):
                values[child.tag] = child.text

        # Process attribute with matching attribute names.
        for attr_name in attr_names:
            for val in nasl.xpath(f'attributes/attribute[name="{attr_name}"]/value'):
                values[attr_name] = val.text

        csv_writer.writerow(values)

输出(test.csv)

"script_id","script_name","script_family","solution","description","synopsis","cvss_base_score","plugin_type","stig_severity"
"101028","Fedora 25 : xen (2017-c3149b5fcb)","Fedora Local Security Checks","Unknown","Unknown","Unknown","10.0","local","I"