如何在xpath python中转换变量

时间:2017-10-04 18:01:54

标签: python xpath web-scraping scrapy web-crawler

    from __future__ import unicode_literals

import requests
from scrapy.selector import Selector

def fetch_page(url):
    r = requests.get(url)
    return r.text

def upso_list_from_listpage(url):
    html = fetch_page(url)
    sel = Selector(text=html)
    upso_lists = []
    for i in range ( 1, 31 ):
        upso_list = sel.xpath ( '//*[@id="List_['"+i+"']"]/div[4]/span/a/text()').extract()
        upso_list.append ( upso_lists )
    #upso_list = sel.xpath('//*[@id="List_0"]/div[4]/span/a/text()').extract()
    #upso_list = sel.css('.nclicks(mai.text1)').extract()
    return upso_lists
from pprint import pprint
pprint(upso_list_from_listpage('http://mokorea.com/ajax/makeBusinessList.php?_method=get&grp=&major=4&cat=&keyword=&hchar=&recom=&page=1&dist=500&gLat=59.8767&gLong=30.4016'))

我想读多行。任何人都可以帮助我将xpath更改为变量吗?我的xpath增加如(List_0,List_1,List_2,List_3 ......)

1 个答案:

答案 0 :(得分:0)

您的代码将是:

locations = [1,2,3]
flight_results   = self.flights.where(from_id: locations).or(self.flights.where(to_id: locations))
waypoint_results = self.waypoints.where(location_id: locations)
search_results = flight_results + waypoint_results
search_results