Question

我需要用Python创建一个程序，该程序可以解析.html文件中的所有URL，并像这样打印出所有标记和链接：

meta: https://someurl.com
a: https://someurl.com
link: css/bootstrap.min.css
script: https://somescript.js

目前，我拥有的是

from html.parser import HTMLParser
import re

class HeadParser(HTMLParser):

def handle_starttag(self, tag, attrs):
    #use re.findall to get all the links
    links = re.findall("http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\), ]|(?:%[0-9a-fA-F][0-9a-fA-F]))+", website)
    for url in links:
        print("{0}: {1}".format(tag, url))

website = open("./head.html").read()            
HeadParser().feed(website)

回到我身边

head: https://scooptacular.net
head: https://scooptacular.net/img/uploaded/379d05029c0d84618c70ac037a25fd88.jpg
head: https://scooptacular.net/img/uploaded/4baaa58a1a37fd3da3e4e78caf366b7f.jpg
head: https://fonts.googleapis.com/css?family=Montserrat:400,700
head: https://fonts.googleapis.com/css?family=Kaushan+Script' rel='stylesheet' type='text/css'>
head: https://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic,700italic' rel='stylesheet' type='text/css'>
head: https://fonts.googleapis.com/css?family=Roboto+Slab:400,100,300,700' rel='stylesheet' type='text/css'>
head: https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js
head: https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js
meta: https://scooptacular.net
meta: https://scooptacular.net/img/uploaded/379d05029c0d84618c70ac037a25fd88.jpg
meta: https://scooptacular.net/img/uploaded/4baaa58a1a37fd3da3e4e78caf366b7f.jpg
meta: https://fonts.googleapis.com/css?family=Montserrat:400,700
meta: https://fonts.googleapis.com/css?family=Kaushan+Script' rel='stylesheet' type='text/css'>
meta: https://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic,700italic' rel='stylesheet' type='text/css'>
meta: https://fonts.googleapis.com/css?family=Roboto+Slab:400,100,300,700' rel='stylesheet' type='text/css'>
meta: https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js
meta: https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js
meta: https://scooptacular.net
meta: https://scooptacular.net/img/uploaded/379d05029c0d84618c70

如您所见，它为我返回了每个标签的链接，甚至是重复的标签，并且不返回任何本地文件链接。我的代码有什么问题？

编辑：

我正在使用的html是：

<head>
<meta property="og:url" content="https://scooptacular.net" />    
<meta property="og:image" content="https://scooptacular.net/img/uploaded/379d05029c0d84618c70ac037a25fd88.jpg" />
<meta property="og:image" content="https://scooptacular.net/img/uploaded/4baaa58a1a37fd3da3e4e78caf366b7f.jpg" />

<link href="css/bootstrap.min.css" rel="stylesheet">

<link href="css/agency.css" rel="stylesheet">

<link href="font-awesome-4.1.0/css/font-awesome.min.css" rel="stylesheet" type="text/css">

        <link href="https://fonts.googleapis.com/css?family=Montserrat:400,700" rel="stylesheet" type="text/css">
    <link href='https://fonts.googleapis.com/css?family=Kaushan+Script' rel='stylesheet' type='text/css'>
    <link href='https://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic,700italic' rel='stylesheet' type='text/css'>
    <link href='https://fonts.googleapis.com/css?family=Roboto+Slab:400,100,300,700' rel='stylesheet' type='text/css'>


<link href="css/bootstrap-formhelpers.min.css" rel="stylesheet" media="screen">

    <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
    <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
</head>

Answer 1

主要问题是每个标签都被调用handle_starttag，并且每次调用时您都在 entire 页面中搜索与您的正则表达式匹配的内容，而不仅是您开启（您要传递给re.findall的第二个参数是website）。

在这里我根本看不到为什么需要使用正则表达式。为什么不仅仅依赖于标签是否具有href，src或content属性：

from html.parser import HTMLParser


class HeadParser(HTMLParser):

    def handle_starttag(self, tag, attrs):
        for attr in attrs:
            if attr[0] in ['href', 'src', 'content']:
                print('{0}: {1}'.format(tag, attr[1]))


website = open("./head.html").read()
HeadParser().feed(website)

输出：

meta: https://scooptacular.net
meta: https://scooptacular.net/img/uploaded/379d05029c0d84618c70ac037a25fd88.jpg
meta: https://scooptacular.net/img/uploaded/4baaa58a1a37fd3da3e4e78caf366b7f.jpg
link: css/bootstrap.min.css
link: css/agency.css
link: font-awesome-4.1.0/css/font-awesome.min.css
link: https://fonts.googleapis.com/css?family=Montserrat:400,700
link: https://fonts.googleapis.com/css?family=Kaushan+Script
link: https://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic,700italic
link: https://fonts.googleapis.com/css?family=Roboto+Slab:400,100,300,700
link: css/bootstrap-formhelpers.min.css
script: https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js
script: https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js

如何使用python，html.parser和regex提取网址

1 个答案: