从TXT文件

时间:2018-04-28 17:06:13

标签: xml awk data-conversion

我有下面的输入txt文件,我正在尝试生成下面的XMl文件。我试图用awk制作它但是 我想我正在重新发明轮子。你是怎么建议我这样做的?感谢

输入txt文件(示例,此输入可能更大)

Usw 1:1 Desktop
Usw 1:2 Netbooks
Usw 1:3 Servers, mainframes and supercomputers
Usw 1:4 Smart devices
Usw 1:5 Embedded devices
Usw 1:6 Gaming
Usw 1:7 Specialized uses
Usw 2:1 Precursors
Usw 2:2 Creation
Usw 2:5 Naming
Usw 2:6 Commercial and popular uptake
Usw 2:9 Current development
Des 1:1 User interface
Des 1:2 Video input infrastructure
Des 1:3 Hardware
Des 2:1 Community
Des 2:2 Programming on Linux

所需的xml文件

<?xml version="1.0" encoding="utf-8"?>

<XMLRT xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="SomeSchema.xsd" bename="The name" status="v" version="1.4" revision="1" type="x-rt">
<INTRO>
    <title>Some title</title>
    <creator>
    </creator>
    <subject>Some subject</subject>
    <description>Some description</description>
    <date>2010-05-12</date>
    <type>Some text</type>
</INTRO>
<RTBLOCK bname="Usw" bnumber="1" bsname="1U">
    <CTR cnumber="1">
    <ES vnumber="1">Desktop</ES>
    <ES vnumber="2">Netbooks</ES>
    <ES vnumber="3">SerES, mainframes and supercomputers</ES>
    <ES vnumber="4">Smart devices</ES>
    <ES vnumber="5">Embedded devices</ES>
    <ES vnumber="6">Gaming</ES>
    <ES vnumber="7">Specialized uses</ES>
    </CTR>
    <CTR cnumber="2">
    <ES vnumber="1">Precursors</ES>
    <ES vnumber="2">Creation</ES>
    <ES vnumber="5">Naming</ES>
    <ES vnumber="6">Commercial and popular uptake</ES>
    <ES vnumber="9">Current development</ES>
    </CTR>
</RTBLOCK>
<RTBLOCK bname="Des" bnumber="1" bsname="1D">
    <CTR cnumber="1">
    <ES vnumber="1">User interface</ES>
    <ES vnumber="2">Video input infrastructure</ES>
    <ES vnumber="3">Hardware</ES>
    </CTR>
    <CTR cnumber="2">
    <ES vnumber="1">Community</ES>
    <ES vnumber="2">Programming on Linux</ES>
    </CTR>
</RTBLOCK>
</XMLRT>

2 个答案:

答案 0 :(得分:5)

只是为了表明您不需要一个支持XML的工具来为任何特定目的生成您需要的特定XML ,这是为您的示例执行此操作的一种方法:

$ cat tst.awk
BEGIN {
    print    "<?xml version=\"1.0\" encoding=\"utf-8\"?>"
    print    ""
    print    "<XMLRT xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:noNamespaceSchemaLocation=\"SomeSchema.xsd\" bename=\"The name\" status=\"v\" version=\"1.4\" revision=\"1\" type=\"x-rt\">"
    print    "<INTRO>"
    print    "    <title>Some title</title>"
    print    "    <creator>"
    print    "    </creator>"
    print    "    <subject>Some subject</subject>"
    print    "    <description>Some description</description>"
    print    "    <date>2010-05-12</date>"
    print    "    <type>Some text</type>"
    print    "</INTRO>"

    rtBeg  = "<RTBLOCK bname=\"%s\" bnumber=\"1\" bsname=\"1%s\">\n"
    ctrBeg = "    <CTR cnumber=\"%d\">\n"
    esBody = "    <ES vnumber=\"%d\">%s</ES>\n"
    ctrEnd = "    </CTR>\n"
    rtEnd  = "</RTBLOCK>\n"
    xmlEnd = "</XMLRT>\n"
}
{
    bname = $1

    split($2,tmp,/:/)
    cnum = tmp[1]
    vnum = tmp[2]

    text = $0
    sub(/([^[:space:]]+[[:space:]]+){2}/,"",text)
}

bname != prevBname {
    if (prevCnum  != "") printf ctrEnd
    if (prevBname != "") printf rtEnd
    printf rtBeg, bname, substr(bname,1,1)
    prevCnum = ""
    prevBname = bname
}

cnum != prevCnum {
    if (prevCnum != "") printf ctrEnd
    printf ctrBeg, cnum
    prevCnum = cnum
}

{ printf esBody, vnum, text }

END {
    if (prevCnum  != "") printf ctrEnd
    if (prevBname != "") printf rtEnd
    printf xmlEnd
}

$ awk -f tst.awk file
<?xml version="1.0" encoding="utf-8"?>

<XMLRT xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="SomeSchema.xsd" bename="The name" status="v" version="1.4" revision="1" type="x-rt">
<INTRO>
    <title>Some title</title>
    <creator>
    </creator>
    <subject>Some subject</subject>
    <description>Some description</description>
    <date>2010-05-12</date>
    <type>Some text</type>
</INTRO>
<RTBLOCK bname="Usw" bnumber="1" bsname="1U">
    <CTR cnumber="1">
    <ES vnumber="1">Desktop</ES>
    <ES vnumber="2">Netbooks</ES>
    <ES vnumber="3">Servers, mainframes and supercomputers</ES>
    <ES vnumber="4">Smart devices</ES>
    <ES vnumber="5">Embedded devices</ES>
    <ES vnumber="6">Gaming</ES>
    <ES vnumber="7">Specialized uses</ES>
    </CTR>
    <CTR cnumber="2">
    <ES vnumber="1">Precursors</ES>
    <ES vnumber="2">Creation</ES>
    <ES vnumber="5">Naming</ES>
    <ES vnumber="6">Commercial and popular uptake</ES>
    <ES vnumber="9">Current development</ES>
    </CTR>
</RTBLOCK>
<RTBLOCK bname="Des" bnumber="1" bsname="1D">
    <CTR cnumber="1">
    <ES vnumber="1">User interface</ES>
    <ES vnumber="2">Video input infrastructure</ES>
    <ES vnumber="3">Hardware</ES>
    </CTR>
    <CTR cnumber="2">
    <ES vnumber="1">Community</ES>
    <ES vnumber="2">Programming on Linux</ES>
    </CTR>
</RTBLOCK>
</XMLRT>

以上任何UNIX框上的任何shell中的任何POSIX awk都可以高效,健壮,可移植地工作。

答案 1 :(得分:1)

  

你怎么建议我这样做?

我建议使用像Saxon by Saxonica这样的XSLT-2.0 +处理器来输出所需的XML文件。但是其他XSLT-2.0处理器也能正常工作。

以下XSLT-2.0样式表分两步进行:

  1. 将未解析的文本检索到<xsl:variable>
  2. 通过<xsl:analyze-string>
  3. 使用RegEx解析此(普通)文本变量
  4. 使用<xsl:for-each-group>
  5. 对生成的扁平XML节点进行分组

    因此样式表可能如下所示:

    <xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs">
        <xsl:output method="xml" />    
        <xsl:param name="text-encoding" as="xs:string" select="'utf-8'"/>
        <xsl:param name="text-uri"      as="xs:string" select="'file:///home/kubuntu/Downloads/input.txt'"/>
    
        <xsl:template match="/">
            <XMLRT xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="SomeSchema.xsd" bename="The name" status="v" version="1.4" revision="1" type="x-rt">
                <!-- Step 1 ### get unparsed text -->
                <xsl:variable name="input-text" select="unparsed-text($text-uri, $text-encoding)"/>
                <!-- Step 2 ### Apply RegEx to every line to create <Line...> elements -->
                <xsl:variable name="xmlStepOne">
                    <xsl:for-each select="tokenize($input-text,'&#xa;')">
                        <xsl:if test=".!=''">                  <!-- Skip empty lines -->
                            <xsl:analyze-string select="." regex="([^\s]+)\s([^:]+):([^\s]+)\s(.*)$">
                                <xsl:matching-substring>       <!-- Parse line with RegEx and create <Line...> XML -->
                                    <Line str="{regex-group(1)}" idx1="{regex-group(2)}" idx2="{regex-group(3)}"><xsl:value-of select="regex-group(4)"/></Line>
                                </xsl:matching-substring>
                                <xsl:non-matching-substring>   <!-- Output an error if a line cannot be processed -->
                                    <xsl:message terminate="yes">Error processing line &#xa;<xsl:value-of select="current()"/>&#xa;</xsl:message>
                                </xsl:non-matching-substring>
                            </xsl:analyze-string>                
                        </xsl:if>
                    </xsl:for-each>
                </xsl:variable>
                <!-- Step 3 ### Group the linear flow of <Line...> elements -->
                <xsl:for-each-group select="$xmlStepOne/Line" group-by="@str">
                    <RTBLOCK bname="{current-grouping-key()}" bnumber="1" bsname="{concat('1',substring(current-grouping-key(),1,1))}">
                        <xsl:for-each-group select="current-group()" group-by="@idx1">
                            <xsl:sort select="@idx1" />
                            <CTR cnumber="{@idx1}"> 
                                <xsl:for-each select="current-group()">
                                    <xsl:sort select="@idx2" />
                                    <ES vnumber="{@idx2}"><xsl:value-of select="."/></ES>
                                </xsl:for-each>
                            </CTR>
                        </xsl:for-each-group>
                    </RTBLOCK>
                </xsl:for-each-group>
            </XMLRT>
        </xsl:template>
    
    </xsl:stylesheet>
    

    您可以在开头用两个参数设置输入文件名和编码。

    上面示例文件的输出是:

    <?xml version="1.0" encoding="UTF-8"?>
    <XMLRT xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="SomeSchema.xsd" bename="The name" status="v" version="1.4" revision="1" type="x-rt">
        <RTBLOCK bname="Usw" bnumber="1" bsname="1U">
            <CTR cnumber="1">
                <ES vnumber="1">Desktop</ES>
                <ES vnumber="2">Netbooks</ES>
                <ES vnumber="3">Servers, mainframes and supercomputers</ES>
                <ES vnumber="4">Smart devices</ES>
                <ES vnumber="5">Embedded devices</ES>
                <ES vnumber="6">Gaming</ES>
                <ES vnumber="7">Specialized uses</ES>
            </CTR>
            <CTR cnumber="2">
                <ES vnumber="1">Precursors</ES>
                <ES vnumber="2">Creation</ES>
                <ES vnumber="5">Naming</ES>
                <ES vnumber="6">Commercial and popular uptake</ES>
                <ES vnumber="9">Current development</ES>
            </CTR>
        </RTBLOCK>
        <RTBLOCK bname="Des" bnumber="1" bsname="1D">
            <CTR cnumber="1">
                <ES vnumber="1">User interface</ES>
                <ES vnumber="2">Video input infrastructure</ES>
                <ES vnumber="3">Hardware</ES>
            </CTR>
            <CTR cnumber="2">
                <ES vnumber="1">Community</ES>
                <ES vnumber="2">Programming on Linux</ES>
            </CTR>
        </RTBLOCK>
    </XMLRT>
    

    这种方法的另一个优点是,您可以使用XML / XSLT处理所有内容,因此它可以识别字符编码以及awk或类似的更简单解决方案所涵盖的所有其他内容。