Nokogiri将每一行收集为一行

时间:2012-01-12 21:27:48

标签: ruby-on-rails nokogiri

我运营一个联盟网站,我有代码消耗XML数据源。我编写了一个方法,可以从XML提要中抓取和存储数据,它可以完美地用于我们拥有的60个提要。 Feed中的每一行都是作为@product创建的。

现在输入Linkshare,我用于其他60个Feed的代码对Linkshare Feed不起作用。它没有让@ products.count =几百,而是试图将所有产品加载到@products的一个实例中,所以我得到:

  

Soul Cal奢华心形吊带背心上衣 - WomensOnly背心上衣 - WomensSoul   Cal Deluxe Racer背心上衣 - WomensSoul Cal Deluxe Racer背心   上衣 - WomensSoul Cal豪华赛车背心上衣 - WomensSoul Cal   豪华赛车背部背心上衣 - WomensCrafted Grunge跳线 - WomensMiso   珍珠&火球戒指堆叠 - WomensSoul Cal豪华格子破洞 -   MensSoul Cal Deluxe Hines条纹衬衫 - MensMiso缀饰腰身   连衣裙 - 女装迷人腰带连衣裙 - WomensMiso前领带短裤 -   WomensMiso领带前短裤 - WomensSoul Cal Deluxe弹力紧身裤   长裤 - WomensSoul Cal Deluxe弹力紧身长裤 - 女装灵魂   Cal Deluxe弹力紧身长裤 - 女装Vero Moda Bernice连衣裙 -   WomensMiso蘸下摆长连衣裙 - WomensSoul Cal Deluxe Belted Chinos    - WomensSoul Cal Deluxe弹力紧身长裤 - 女装Miiso珠宝紧身连衣裙 - 女装Miiso晚装饰连衣裙 - 女装Miso Aztec   Tube Skirt - WomensCrafted缀饰领连衣裙 - WomensMiso Spot   连衣裙 - 女装米索现货连衣裙 - 女装“}]

(注意,我刚刚结束了控制台日志,但你明白了我的意思。

我已经进行了大量的测试,这是我们已经有一段时间的问题了。

有人有这样的事吗?

可能是doctypes还是hpricot?

任何建议?

示例Feed:

<?xml version="1.0" encoding="UTF-8"?><merchandiser xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="merchandiser.xsd">
<header><merchantId>36503</merchantId><merchantName>Republic</merchantName><createdOn>2012-01-12/20:21:17</createdOn></header><product product_id="73794" name="Soul Cal Deluxe Borg DLX Zip Through Hoody - Mens" sku_number="73794" manufacturer_name="Soul Cal Deluxe" part_number="73794"><category><primary>Men &gt; Sweats &amp; Hoodies</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.73794&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F73794</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/73794/73794_lg1.jpg</productImage><buy></buy></URL><description><short>Blue, Hoodies, Hand wash only</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">34.99</sale><retail>34.99</retail></price><brand>Soul Cal Deluxe</brand><shipping><cost currency="GBP"><amount>3.95</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.73794&amp;type=15&amp;subid=0</pixel></product>
<product product_id="51118" name="Jack &amp; Jones Dale Jack Jeans - Mens" sku_number="51118" manufacturer_name="Jack&amp;Jones" part_number="51118"><category><primary>Men &gt; Jeans</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.51118&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F51118</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/51118/51118_lg1.jpg</productImage><buy></buy></URL><description><short>Blue, Straight</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">40.00</sale><retail>64.99</retail></price><brand>Jack&amp;Jones</brand><shipping><cost currency="GBP"><amount>3.95</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.51118&amp;type=15&amp;subid=0</pixel></product>
<product product_id="51128" name="Diesel Straight Leg Larkee Jeans - Mens" sku_number="51128" manufacturer_name="Diesel" part_number="51128"><category><primary>Men &gt; Jeans</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.51128&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F51128</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/51128/51128_lg1.jpg</productImage><buy></buy></URL><description><short>Blue, Straight</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">120.00</sale><retail>120.00</retail></price><brand>Diesel</brand><shipping><cost currency="GBP"><amount>0.00</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.51128&amp;type=15&amp;subid=0</pixel></product>
<product product_id="68226" name="Miso Fox Scarf - Womens" sku_number="68226" manufacturer_name="Miso" part_number="68226"><category><primary>Women &gt; Accessories</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.68226&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F68226</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/68226/68226_lg1.jpg</productImage><buy></buy></URL><description><short>Stone, Scarves, Machine washable</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">5.00</sale><retail>19.99</retail></price><brand>Miso</brand><shipping><cost currency="GBP"><amount>3.95</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.68226&amp;type=15&amp;subid=0</pixel></product>
<product product_id="67968" name="Levis Pleat Trucker Jacket - Womens" sku_number="67968" manufacturer_name="Levis" part_number="67968"><category><primary>Women &gt; Coats &amp; Jackets</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.67968&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F67968</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/67968/67968_lg1.jpg</productImage><buy></buy></URL><description><short>Blue, Jackets</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">30.00</sale><retail>84.99</retail></price><brand>Levis</brand><shipping><cost currency="GBP"><amount>3.95</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.67968&amp;type=15&amp;subid=0</pixel></product>
<product product_id="81217" name="G-Star Dean Army Tapered Jeans - Womens" sku_number="81217" manufacturer_name="G-Star" part_number="81217"><category><primary>Women &gt; Jeans</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.81217&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F81217</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/81217/81217_lg1.jpg</productImage><buy></buy></URL><description><short>Black, Straight</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">115.00</sale><retail>115.00</retail></price><brand>G-Star</brand><shipping><cost currency="GBP"><amount>0.00</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.81217&amp;type=15&amp;subid=0</pixel></product>
<product product_id="58377" name="Police 883 Embro Hat - Mens" sku_number="58377" manufacturer_name="Police 883" part_number="58377"><category><primary>Men &gt; Accessories</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.58377&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F58377</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/58377/58377_lg1.jpg</productImage><buy></buy></URL><description><short>Black, Hats &amp; Beanies</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">17.99</sale><retail>17.99</retail></price><brand>Police 883</brand><shipping><cost currency="GBP"><amount>3.95</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.58377&amp;type=15&amp;subid=0</pixel></product>
<product product_id="58306" name="Diesel Kyle Beanie - Mens" sku_number="58306" manufacturer_name="Diesel" part_number="58306"><category><primary>Men &gt; Accessories</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.58306&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F58306</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/58306/58306_lg1.jpg</productImage><buy></buy></URL><description><short>Black, Hats &amp; Beanies</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">19.99</sale><retail>19.99</retail></price><brand>Diesel</brand><shipping><cost currency="GBP"><amount>3.95</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.58306&amp;type=15&amp;subid=0</pixel></product>
<product product_id="86865" name="Miso Fair Isle Cardigan - Womens" sku_number="86865" manufacturer_name="Miso" part_number="86865"><category><primary>Women &gt; Knitwear</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.86865&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F86865</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/86865/86865_lg1.jpg</productImage><buy></buy></URL><description><short>Stone, Cardigans, Hand wash only</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">49.99</sale><retail>49.99</retail></price><brand>Miso</brand><shipping><cost currency="GBP"><amount>3.95</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.86865&amp;type=15&amp;subid=0</pixel></product>
<product product_id="52947" name="White Label Chinos - Mens" sku_number="52947" manufacturer_name="White Label" part_number="52947"><category><primary>Men &gt; trs</primary><secondary></secondary></category><URL><product>http://click.linksynergy.com/link?id=g*0hRDNOv4M&amp;offerid=215450.52947&amp;type=15&amp;murl=http%3A%2F%2Fwww.republic.co.uk%2Finvt%2F52947</product><productImage>http://www.republic.co.uk/content/ebiz/republic/invt/52947/52947_lg1.jpg</productImage><buy></buy></URL><description><short>Stone, Chinos, Machine washable</short><long></long></description><discount currency="GBP"><amount></amount><type>amount</type></discount><price currency="GBP"><sale begin_date="" end_date="">20.00</sale><retail>20.00</retail></price><brand>White Label</brand><shipping><cost currency="GBP"><amount>3.95</amount><currency>GBP</currency></cost><information></information><availability></availability></shipping><keywords></keywords><upc></upc><m1></m1><pixel>http://ad.linksynergy.com/fs-bin/show?id=g*0hRDNOv4M&amp;bids=215450.52947&amp;type=15&amp;subid=0</pixel></product>

1 个答案:

答案 0 :(得分:0)

一个问题是XML格式错误。 <merchandiser>标记未终止。如果在解析文档后检查errors方法,Nokogiri会告诉您:

doc.errors
[
    [0] #<Nokogiri::XML::SyntaxError:0x10109b918
        attr_reader :code = 77,
        attr_reader :column = 1,
        attr_reader :domain = 1,
        attr_reader :file = nil,
        attr_reader :int1 = 1,
        attr_reader :level = 3,
        attr_reader :line = 12,
        attr_reader :str1 = "merchandiser",
        attr_reader :str2 = nil,
        attr_reader :str3 = nil
    >
]

在文件末尾添加结束</merchandiser>标记,清除了:

doc = Nokogiri::XML(File.read('./test.xml'))
#<Nokogiri::XML::Document:0x1009ae038
    @node_cache = [],
    attr_accessor :errors = [],
    attr_reader :decorators = nil
>

修复后我解析了product

doc.search('product').size
 => 20

<product>次碰撞:

<product>
  <URL>
    <product />
  </URL>
</product>

可能会混淆您的代码。它使我的搜索混淆了报告的20次事件。