我在python中使用urllib2
模块从某些网址(例如http://www.google.co.in/
)的锚标记中获取某种信息,下面是代码
import urllib2
import urlparse
from BeautifulSoup import BeautifulSoup
url = "http://www.google.co.in/"
page = urllib2.urlopen(url)
html = page.read()
page.close()
soup = BeautifulSoup(html)
for tag in soup.findAll('a', href=True):
text = tag.text
tag['href'] = urlparse.urljoin(url, tag['href'])
print ' '.join([text,tag['href']])
结果:
Web History http://www.google.co.in/history/optout?hl=en
Settings http://www.google.co.in/preferences?hl=en
Sign in https://accounts.google.com/ServiceLogin?hl=en&continue=http://www.google.co.in/
Advanced search http://www.google.co.in/advanced_search?hl=en-IN&authuser=0
Language tools http://www.google.co.in/language_tools?hl=en-IN&authuser=0
.......................
现在很好,但我想将信息存储为下面的元组列表
[('Web History','http://www.google.co.in/history/optout?hl=en'),('Settings','http://www.google.co.in/preferences?hl=en'),('Sign in','https://accounts.google.com/ServiceLogin?hl=en&continue=http://www.google.co.in/')................]
所以任何人都可以让我知道我们如何格式化来自for循环的数据如上面的元组列表
答案 0 :(得分:2)
尝试这样的事情:
[(tag.text, urlparse.urljoin(url, tag['href']))
for tag in soup.findAll('a', href=True)]
答案 1 :(得分:0)
您可以尝试创建哈希并从中提取items()
元组,这只是一个黑客攻击:
def __init__(self, *args, **kwargs):
super(IndicatorForm, self).__init__(*args, **kwargs)
d = dir(indicators)
b = {}
for a in d:
b[a] = a
b = b.items()
b.sort()
self.fields["choice"].choices = b
这里dir(指标)是一个数组。