Question

start_urls = http://www.espncricinfo.com/series/18679/scorecard/1144998/australia-vs-india-2nd-odi-india-in-aus-2018-19

我抓取了该网站并提取了比赛结果（获胜队伍），然后产生了球员URL，并且我想打印球员名称和击球风格。我的第一个问题是 1.我无法抽象出球员的打击音。它在<pclass="ciPlayerinformationtxt"><b>Batting style</b> <span>Right-hand bat</span>下。我只能提取文本“ Batting style”。如何提取“ Right-hand bat” 2.我无法将提取的全部数据生成表格。我得到的结果就像

p 所有玩家的链接 http://www.espncricinfo.com/ci/content/player/326434.html

Player_name国家 Alex Carey Australia
印度Kuldeep Yadav
Mohammed Siraj印度
Winning_Team：印度

class ScoreSpider(scrapy.Spider):
    name = 'score'
    allowed_domains = ['espncricinfo.com']

    def parse(self, response):
        Player_URLs=[]

        #got the result

        result= response.xpath('//div[@class="cscore_notes"]/span/text()').extract_first()
        result=result.split(" ")
        Winning_Team =result[0]

        #extracted player ulrs

        Batting_Player_URLs=response.xpath('//div[@class="cell batsmen"]/a/@href').extract()
        Bowling_Player_URLs=response.xpath('//*[@class="scorecard-section bowling"]/table/tbody/tr/td/a/@href').extract()

        #added to a list

        Player_URLs.extend(Batting_Player_URLs)
        Player_URLs.extend(Bowling_Player_URLs)
        for p in Player_URLs:
            yield Request(p,callback=self.parse_players,meta={'p':p})
            yield{'Winning_Team':Winning_Team}

    def parse_players(self,response):
        Player_name=response.xpath('//div[@class="ciPlayernametxt"]/div/h1/text()').extract_first()
        Country=response.xpath('//div[@class="ciPlayernametxt"]/div/h3/b/text()').extract_first()

        #this wont give the batting style but the 'batting style' as text
        Batting_style=response.xpath('//div[@class="ciPlayerinformationtxt"]/p/text()').extract_first()

        yield{'Player_name':Player_name,
              'Country':Country, 
              'Batting_style':Batting_style}

我想要的是将提取的数据作为单个表，并且我想避免重复。

          yield{'Winning_Team':Winning_Team,
                'Player_name':Player_name,
                'Country':Country,
                'Batting_style':Batting_style}

预先感谢

Answer 1

您需要调整XPath：

batting_style = response.xpath('//p[@class="ciPlayerinformationtxt"]/b[.="Batting style"]/following-sibling::span[1]/text()').get()

更新

for p in Player_URLs:
    yield Request(p,callback=self.parse_players,meta={'Winning_Team':Winning_Team})

及更高版本：

def parse_players(self,response):
    Winning_Team = response.meta["Winning_Team"]

我正在尝试从espn中提取一些数据作为表格并将其作为列表获取

1 个答案: