Question

Python和BeautifulSoup的新手，我试图从网站上搜集比赛细节，以便在我当地的俱乐部网站上展示。

到目前为止，这是我的代码：

import urllib2
import sys
import os

sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
from BeautifulSoup import BeautifulSoup

# Road
#cyclelab_url='http://www.cyclelab.com/OnLine%20Entries.aspx?type=Road%20Events'

# MTB
cyclelab_url='http://www.cyclelab.com/OnLine%20Entries.aspx?type=Mountain%20Biking%20Events'

response = urllib2.urlopen(cyclelab_url)
html = response.read()

soup = BeautifulSoup(html)
event_names = soup.findAll(attrs= {"class" : "SpanEventName"})
for event in event_names:
    txt = event.find(text=True)
    print txt

event_details = soup.findAll(attrs= {"class" : "TDText"})
for detail in event_details:
    lines=[]
    txt_details = detail.find(text=True)
    print txt_details

这将打印事件名称和事件详细信息，我想要做的是，打印事件名称，然后在其下面显示该事件的事件详细信息。看起来应该很简单，但我很难过。

Answer 1

如果查看页面的结构，您将看到在第一个循环中找到的事件名称被一个表包围，该表包含表中行的单元格对的所有其他有用详细信息。所以，我要做的就是只有一个循环，每次找到事件名称时，查找封闭表并查找其下的所有事件。这似乎工作正常：

soup = BeautifulSoup(html)
event_names = soup.findAll(attrs= {"class" : "SpanEventName"})
for event in event_names:
    txt = event.find(text=True)
    print "Event name: "+txt.strip()
    # Find each parent in turn until we find the table that encloses
    # the event details:
    parent = event.parent
    while parent and parent.name != "table":
        parent = parent.parent
    if not parent:
        raise Exception, "Failed to find a <table> enclosing the event"
    # Now parent is the table element, so look for every
    # row under that table, and then the cells under that:
    for row in parent.findAll('tr'):
        cells = row.findAll('td')
        # We only care about the rows where there is a multiple of two
        # cells, since these are the key / value pairs:
        if len(cells) % 2 != 0:
            continue
        for i in xrange(0,len(cells),2):
            key_text = cells[i].find(text=True)
            value_text = cells[i+1].find(text=True)
            if key_text and value_text:
                print "  Key:",key_text.strip()
                print "  Value:",value_text.strip()

输出如下：

Event name: Columbia Grape Escape 2011
  Key: Category:
  Value: Mountain Biking Events
  Key: Event Date:
  Value: 4 March 2011 to 6 March 2011
  Key: Entries Close:
  Value: 31 January 2011 at 23:00
  Key: Venue:
  Value: Eden on the Bay, Blouberg
  Key: Province:
  Value: Western Cape
  Key: Distance:
  Value: 3 Day, 3 Stage Race (228km)
  Key: Starting Time:
  Value: -1:-1
  Key: Timed By:
  Value: RaceTec
Event name: Investpro MTB Race 2011
  Key: Category:
  Value: Mountain Biking Events
  Key: Event Date:
  Value: 5 March 2011
  Key: Entries Close:
  Value: 25 February 2011 at 23:00

......等等。

Answer 2

更新：Mark Longair有正确/更好的答案。见评论。

代码从上到下执行。因此，在您的代码中，首先打印所有事件，然后打印详细信息。您必须将代码“编织”在一起，意味着每个事件，打印所有细节，然后移动到下一个事件。尝试这样的事情：

[....]
event_names = soup.findAll(attrs= {"class" : "SpanEventName"})
event_details = soup.findAll(attrs= {"class" : "TDText"})
for event in event_names:
       txt = event.find(text=True)
       print txt
    for detail in event_details:
        txt_details = detail.find(text=True)
        print txt_details

进一步改进：您可以使用.strip（）删除所有空格和换行符。例如：text_details = detail.find(text=True).strip()。

BeautifulSoup从远程站点抓取细节以在本地站点上显示

2 个答案: