Question

我正在尝试解析看起来像xml但不是的文件。 Actualy它是从ASN1格式转换的CRD的可读版本。它看起来像这样：

<PIN rowNum="1">
<CgPa tag="3100.2.960.51" value="1">
<data tag="3100.2.962.56" name="cgPASubscriberIdentifier" value="50212000000089804"/>
<data tag="3100.2.962.60" name="cgPaRoaming" value="1"/>
</CgPa>
<AAA_Common tag="3100.2.960.1" value="1">
<data tag="3100.2.962.12" name="sigSleeId" value="watbf102"/>
<data tag="3100.2.962.34" name="scpAddress" value="48602888950"/>
</AAA_Common>
<evt tag="3100.2.134.28" name="unsupported" value="0"/>
<data tag="3100.2.112.1" name="eventDateTime" value="07/05/2014 19:45:18"/>
<data tag="3100.2.137.4" name="inTriggeringKey" value="0048662221827"/>
<evt tag="3100.2.137.5" name="typeINTriggeringKey" value="1"/>
<CustomerDomain tag="3100.2.134.1" value="1">
<data tag="3100.2.133.1" name="ordinaryClientId" value="50212000000089804"/>
<data tag="3100.2.105.1" name="customerServiceName" value="SO_TT_Roam_Voice"/>
<AccountDomain tag="3100.2.134.3" value="1">
<data tag="3100.2.104.4" name="accountIdentifier" value="50212000000089804"/>
<data tag="3100.2.100.1" name="subscriberType" value="1"/>
<evt tag="3100.2.139.3" name="unsupported" value="0"/>
<TariffDomain tag="3100.2.134.11" value="1">
<data tag="3100.2.106.10" name="tariffPlanNameVersion" value="TT_VOI_R_1_PL_1A_0_RoamB - 2_TCA"/>
</TariffDomain>
<TariffDomain tag="3100.2.134.11" value="1">
<data tag="3100.2.106.10" name="tariffPlanNameVersion" value="TT_VOI_R_1_PL_1A_0_Main - 2_TCA"/>
<data tag="3100.2.106.1" name="tariffPlanName" value="TT_VOI_R_1_PL_1A_0_Main"/>
<evt tag="3100.2.139.9" name="tariffCost" value="1013"/>
<evt tag="3100.2.139.10" name="tariffCostVat" value="1013"/>
<evt tag="3100.2.140.7" name="eventQuantityPerTariff1" value="614"/>
<evt tag="3100.2.142.11" name="usedQuantityPerTariff1" value="614"/>
</TariffDomain>
<evt tag="3100.2.134.29" name="unsupported" value="1"/>
<data tag="3100.2.124.45" name="unsupported" value="07/05/2014 19:45:18"/>
<evt tag="3100.2.139.35" name="unsupported" value="495"/>
<data tag="3100.2.24.11" name="unsupported" value="84490"/>
<evt tag="3100.2.134.30" name="unsupported" value="1"/>
</AccountDomain>
</CustomerDomain>
</PIN>

每条记录的主标记是PIN，但子标记可以按随机顺序显示或根本不显示。猪的xml案例的典型解决方案是使用piggybank函数XMLLoader。但它假设标签的顺序是不变的。否则我们无法将其置于模式中。我看到REGEXP每行的唯一解决方案，并使用名称和值并使用map[]。但是，在我的示例中，标签看起来更像是TariffDomain？我该如何处理？

问候
的Pawel

Answer 1

我只想提出一个想法，如果这对您有用，请告诉我的算法
1.解析每一行并使用REGEX取名称和值 2.删除所有空字符串
3.根据键
对所有行进行分组 4.使用多个值作为包映射每个键

PigScript: A = LOAD 'input.txt' as line; B = FOREACH A GENERATE FLATTEN(REGEX_EXTRACT_ALL(line,'.*name="(.*)"\\s+value="(.*)".*')) as(mykey:chararray,myvalue:chararray); C = FILTER B BY mykey IS NOT NULL; D = GROUP C BY mykey; E = FOREACH D GENERATE TOMAP(group,C.myvalue); dump E; Output: ([sigSleeId#{(watbf102)}]) ([scpAddress#{(48602888950)}]) ([tariffCost#{(1013)}]) ([cgPaRoaming#{(1)}]) ([unsupported#{(1),(0),(0),(1),(07/05/2014 19:45:18),(495),(84490)}]) ([eventDateTime#{(07/05/2014 19:45:18)}]) ([tariffCostVat#{(1013)}]) ([subscriberType#{(1)}]) ([tariffPlanName#{(TT_VOI_R_1_PL_1A_0_Main)}]) ([inTriggeringKey#{(0048662221827)}]) ([ordinaryClientId#{(50212000000089804)}]) ([accountIdentifier#{(50212000000089804)}]) ([customerServiceName#{(SO_TT_Roam_Voice)}]) ([typeINTriggeringKey#{(1)}]) ([tariffPlanNameVersion#{(TT_VOI_R_1_PL_1A_0_Main - 2_TCA),(TT_VOI_R_1_PL_1A_0_RoamB - 2_TCA)}]) ([usedQuantityPerTariff1#{(614)}]) ([eventQuantityPerTariff1#{(614)}]) ([cgPASubscriberIdentifier#{(50212000000089804)}])

如何将伪xml转换为扁平结构？

1 个答案: