Question

我开始使用Goutte从网站获取我需要的信息，并且它的精彩，它节省了我的大量时间和麻烦。但另一方面，有时出现异常现象，我不知道是什么导致了它们。所以这是我正在抓的一页： http://lifecenter.sgst.cn/hit/search.jsp?key1=%27a%27&key2=%27b%27&key3=%27c%27&key4=%27d%27&key5=C0305

我一直在浏览这个网站，关注链接，网站上的每个页面都有自己的表格。这个表看起来非常简单，其中一些是HTML：

<table align="center" border="1" cellpadding="4" cellspacing="1" width="927">
<tbody><tr bgcolor="#3399FF">
<th align="center">ENTRY</th>
<th align="center">COMPOUND</th>
<th align="center">HERB</th>
<th align="center">TARGET ID</th>
<th align="center">TARGET NAME</th>
<th align="center">TARGET TYPE</th>
</tr>
</tbody><tbody id="table2">


<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin p60 ATPase-containing subunit A1">T0595</a></td>
<td align="center">Katanin p60 ATPase-containing subunit A1</td>
<td align="center">Direct Target</td>
</tr>

<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0596">HIT001883</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0596&amp;protein=Adenosine receptor A2a">T0596</a></td>
<td align="center">Adenosine receptor A2a</td>
<td align="center">Direct Target</td>
</tr>

<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0040">HIT001885</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0040&amp;protein=Nitric oxide synthase, inducible">T0040</a></td>
<td align="center">Nitric oxide synthase, inducible</td>
<td align="center">Direct Target</td>
</tr>

<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0054">HIT001884</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0054&amp;protein=Prostaglandin G/H synthase 2">T0054</a></td>
<td align="center">Prostaglandin G/H synthase 2</td>
<td align="center">Indirect Target</td>
</tr>

</tbody>
</table>

所以我做的第一件事是选择<tbody>，因为我需要的所有数据都在那里：

    $tbody = $compoundPage->filter('tbody#table2');

当我跑步时，一切看起来都不错：

> exit(dump($compoundPage->filter('tbody#table2')->html()));

以下是我得到的：

<tr>\n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>\r\n
    <td align="center">linalool</td>\r\n
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>\r\n
    <td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin%20p60%20ATPase-containing%20subunit%20A1">T0595</a></td>\r\n
    <td align="center">Katanin p60 ATPase-containing subunit A1</td>\r\n
    <td align="center">Direct Target</td>\r\n
</tr>\n
<tr>\n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0596">HIT001883</a></td>\r\n
    <td align="center">linalool</td>\r\n
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>\r\n
    <td align="center"><a href="Protein_ID.jsp?pid=T0596&amp;protein=Adenosine%20receptor%20A2a">T0596</a></td>\r\n
    <td align="center">Adenosine receptor A2a</td>\r\n
    <td align="center">Direct Target</td>\r\n
</tr>\n
<tr>\n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0040">HIT001885</a></td>\r\n
    <td align="center">linalool</td>\r\n
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Radix Bupleuri chinensis(</td>\r\n
    <td align="center"><a href="Protein_ID.jsp?pid=T0040&amp;protein=Nitric%20oxide%20synthase,%20inducible">T0040</a></td>\r\n
    <td align="center">Nitric oxide synthase, inducible</td>\r\n
    <td align="center">Direct Target</td>\r\n
</tr>\n
<tr>\n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0054">HIT001884</a></td>\r\n
    <td align="center">linalool</td>\r\n
    <td align="center">Ardisiae japonicae</td>\r\n
    <td align="center"><a href="Protein_ID.jsp?pid=T0054&amp;protein=Prostaglandin%20G/H%20synthase%202">T0054</a></td>\r\n
    <td align="center">Prostaglandin G/H synthase 2</td>\r\n
    <td align="center">Indirect Target</td>\r\n
    </tr>

我看到4行。

但现在我需要遍历行，所以我必须更进一步：

$tbody = $compoundPage->filter('tbody#table2 > tr');

我跑的时候：exit(dump($compoundPage->filter('tbody#table2 > tr')->html()));

它只输出一行：

<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>\r\n
<td align="center">linalool</td>\r\n
<td align="center">Ardisiae japonicae(ai di cha|矮地茶);</td>\r\n
<td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin%20p60%20ATPase-containing%20subunit%20A1">T0595</a></td>\r\n
<td align="center">Katanin p60 ATPase-containing subunit A1</td>\r\n
<td align="center">Direct Target</td>\r\n

但是当我计算这样的行时：

退出（转储（$ compoundPage-＆gt;过滤器（'tbody＃table2＆gt; tr'） - ＆gt; count（）））;

它表示有4行。

当我遍历行时出现问题，如下所示：

    $data = $rows->each(function($row,$i) use ($client) {
      $tds = $row->filter('td');

      $keys = array('entry','compound','herb','target_id','target_name','target_type');

      foreach($keys as $td_i => $key) {
          $data[$key] = $tds->eq($i)->text();
      }

  });

首先，当我计算td元素时，它表示只有1.如果我尝试对td节点实际执行任何操作，我会收到此错误： https://gyazo.com/94042ba2e3101ba918a71cc9e8e3be5a

如果我这样做：

出口（转储（$行级别＆GT;过滤器（ 'TD'）））;

在循环内部，它告诉我有24个节点：

Crawler {#1121 ▼
  #uri: "http://lifecenter.sgst.cn/hit/search.jsp?key1='a'&key2='b'&key3='c'&key4='d'&key5=C0305"
   -defaultNamespacePrefix: "default"
   -namespaces: []
   -baseHref: "http://lifecenter.sgst.cn/hit/search.jsp?key1='a'&key2='b'&key3='c'&key4='d'&key5=C0305"
   -document: DOMDocument {#52 ▶}
   -nodes: array:24 [▼
     0 => DOMElement {#1123 ▶}
     1 => DOMElement {#1124 ▶}
     2 => DOMElement {#1125 ▶}
     3 => DOMElement {#1126 ▶}
     4 => DOMElement {#1127 ▶}
     5 => DOMElement {#1128 ▶}
     6 => DOMElement {#1129 ▶}
     7 => DOMElement {#1130 ▶}
     8 => DOMElement {#1131 ▶}
     9 => DOMElement {#1132 ▶}
     ... (omitted the rest cuz I have to do this indentation manually, its so time consuming.

  ]
  -isHtml: true

Answer 1

我得到了它的工作，我补充说：

$ rows = $ compoundPage-＆gt; filter（＆＃39; tbody＃table2＆＃39;） - ＆gt; filter（＆＃39; tr＆＃39;）;

而不是：

$ rows = $ compoundPage-＆gt; filter（＆＃39; tbody＃table2＆gt; tr＆＃39;）;

现在它正常工作。我仍然不知道问题是什么，因为这两种选择方式应该是相同的，我很确定我之前尝试过这种方法。我在循环表中遇到了很多问题，我发现它们之间肯定存在差异：

$ rows = $ compoundPage-＆gt; filter（＆＃39; tbody＃table2＆gt; tr＆＃39;）;

和

$ rows = $ compoundPage-＆gt; filter（＆＃39; tbody＃table2＆＃39;） - ＆gt; filter（＆＃39; tr＆＃39;）;

我已经看到CSS选择方法多次破坏脚本。

Answer 2

有关“如何使用symfony dom crawler将html表解析为数组”的问题，请参阅此答案here

Goutte - Guzzle / DomCrawler - 从HTML表中删除 - 很多复杂情况

2 个答案: