Question

我想采取网站的三个组件，首先是Ipadress，端口和protoco，我在python中使用HTMLParser来解决，但标签中的下面的代码没有attrisbute和值

Structure

那么如何获得IPadress和端口的所有3个组件，protoco。

Answer 1

尝试使用 lxml ：

from lxml import html

HTML_CODE = """<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>proxy-list</title>
</head>
    <body>
        <table>
            <tr>
                <td>192.168.1.10</td>     
                <td>HTTP1</td>
                <td>Vietnam</td>
                <td>8080</td>
                <td>a</td>
                <td>%d1</td>
            </tr>
            <tr>
                <td>10.25.100.10</td>
                <td>HTTPS2</td>
                <td>Campuchia</td>
                <td>3214</td>
                <td>b</td>
                <td>%d2</td>
            </tr>
            <tr>
                <td>203.25.10.110</td>
                <td>HTTP3</td>
                <td>ThaiLan</td>
                <td>3215</td>
                <td>c</td>
                <td>%d3</td>
            </tr>
            <tr>
                <td>220.155.10.13</td>
                <td>HTTP4</td>
                <td>Itali</td>
                <td >1000</td>
                <td>d</td>
                <td>%d4</td>
            </tr>
            <tr>
                <td>220.155.10.13</td>
                <td>HTTP5</td>
                <td>Itali</td>
                <td >1000</td>
                <td>d</td>
                <td>%d4</td>
            </tr>
            <tr>
                <td>220.155.10.13</td>
                <td>HTTP6</td>
                <td>Itali</td>
                <td >1000</td>
                <td>d</td>
                <td>%d4</td>
            </tr>
        </table>
    </body>
</html>"""

tree = html.fromstring(HTML_CODE)

for block in tree.xpath('//table/tr'):
    ip, protocol, _, port, _, _ = [x.strip() for x in block.xpath('.//text()') if x.strip() not in ""]
    print "IP: {}".format(ip)
    print "Protocol: {}".format(protocol)
    print "Port: {}\n".format(port)

<强>输出：

IP: 192.168.1.10
Protocol: HTTP1
Port: 8080

IP: 10.25.100.10
Protocol: HTTPS2
Port: 3214

IP: 203.25.10.110
Protocol: HTTP3
Port: 3215

IP: 220.155.10.13
Protocol: HTTP4
Port: 1000

IP: 220.155.10.13
Protocol: HTTP5
Port: 1000

IP: 220.155.10.13
Protocol: HTTP6
Port: 1000

Answer 2

您必须自己添加标签。我同意Andres lxml更适合这一点，但使用HTMLParser可以创建一个TableParser类，它输出一个包含每个表行字典的嵌套数组。

import HTMLParser
html = """<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>proxy-list</title>
</head>
    <body>
        <table>
            <tr>
                <td>192.168.1.10</td>     
                <td>HTTP1</td>
                <td>Vietnam</td>
                <td>8080</td>
                <td>a</td>
                <td>%d1</td>
            </tr>
            <tr>
                <td>10.25.100.10</td>
                <td>HTTPS2</td>
                <td>Campuchia</td>
                <td>3214</td>
                <td>b</td>
                <td>%d2</td>
            </tr>       
        </table>
    </body>
</html>"""

class TableParser(HTMLParser.HTMLParser):
   def __init__(self,tags):
       self.tags=tags
       HTMLParser.HTMLParser.__init__(self)
       self.in_td = False  

   def handle_starttag(self, tag, attrs):
        if tag == 'td':           
           self.in_td = True           
        if tag == 'tr':
            self.count = 0
            self.row = {}

   def handle_data(self, data):
       if self.in_td:           
           self.row[self.tags[self.count]] = data
           self.count = self.count +1          

   def handle_endtag(self, tag):
      self.in_td = False
      if tag == 'tr':
            self.table.append(self.row)

   def feed_tags(self,html,tags):  
       self.table=[]       
       self.feed(html)
       return self.table

tags=  ['ip','protocol','country','port','field1','field2'] 
p = TableParser(tags)    
table = p.feed_tags(html,tags)
for row in table:
    print row['ip']+','+row['protocol']+','+row['port']

从网站HTMLParser Python获取数据

2 个答案: