Question

我已经使用Python Beautifulsoup（https://en.wikipedia.org/wiki/Districts_of_Hong_Kong）废弃了Wikipedia表。但是除了提供的数据（即人口，面积，密度和区域）之外，我想获取每个地区的位置坐标。数据应该来自每个地区的另一页（表上有超链接）。

以第一区“中西部区”为例，在页面上可以找到DMS坐标（22°17′12″ N 114°09′18″ E）。通过进一步单击链接，我可以获得十进制坐标（22.28666，114.15497）。

因此，是否可以为每个地区创建具有纬度和经度的表格？

对于编程世界来说是新手，对不起，如果这个问题很愚蠢...

参考：

DMS坐标：https://en.wikipedia.org/wiki/Central_and_Western_District

十进制坐标：https://tools.wmflabs.org/geohack/geohack.php?pagename=Central_and_Western_District&params=22.28666_N_114.15497_E_type:adm2nd_region:HK

Answer 1

     BinaryData modelData = AssetUtils.loadAsset("raw/fcd_auto.nml");

结果：

import requests
from bs4 import BeautifulSoup

res = requests.get('https://en.wikipedia.org/wiki/Districts_of_Hong_Kong')
result = {}
soup = BeautifulSoup(res.content,'lxml')
tables = soup.find_all('table',{'class':'wikitable'})
table = tables[0].find('tbody')
districtLinks = table.find_all('a',href=True)

for link in districtLinks:
    if link.getText() in link.attrs.get('title','') or link.attrs.get('title','') in link.getText():
        district = link.attrs.get('title','')
        if district:
            url = link.attrs.get('href', '')
        else:
            continue
    else:
        continue
    try:
        res = requests.get("https://en.wikipedia.org/{}".format(url))
    except:
        continue
    else:
        soup = BeautifulSoup(res.content, 'lxml')
        try:
            tables = soup.find_all('table',{'class':'infobox geography vcard'})
            table = tables[0].find('tbody')
        except:
            continue
        else:
            for row in table.find_all('tr',{'class':'mergedbottomrow'}):
                geoLink = row.find('span',{'class': 'geo'}) # 'plainlinks nourlexpansion'
                locationSplit = geoLink.getText().split("; ")
                result.update({district : {"Latitude ": locationSplit[0], "Longitude":locationSplit[1]}})

print(result)

Answer 2

创建使用区域链接到页面来加载它并使用BeautifulSoup搜索纬度和经度以及链接中十进制坐标的函数。

然后将其作为列表返回到主函数，并与其他信息一起添加到行中。

import requests
from bs4 import BeautifulSoup as BS
import re

def parse_district(url):
    r = requests.get(url)

    soup = BS(r.text, 'html.parser')

    link = soup.find('a', {'href': re.compile('//tools.wmflabs.org/.*')})

    item = link['href'].split('params=')[1].split('type:')[0].replace('_', ' ').strip()
    #print(item)

    items = link.find_all('span', {'class':('latitude', 'longitude')})

    #print('>>>', [item] + [i.text for i in items][:3] )

    return [item] + [i.text for i in items]

def main():
    url = 'https://en.wikipedia.org/wiki/Districts_of_Hong_Kong'

    r = requests.get(url)

    soup = BS(r.text, 'html.parser')

    table = soup.find_all('table', {'class': 'wikitable'})
    for row in table[0].find_all('tr'):
        items = row.find_all('td')
        if items:
            row = [i.text.strip() for i in items]

            link = 'https://en.wikipedia.org' + items[0].a['href']
            data = parse_district(link)

            row += data
            print(row)
main()

结果

['Central and Western', '中西區', '244,600', '12.44', '19,983.92', 'Hong Kong Island', '22.28666 N 114.15497 E', '22°17′12″N', '114°09′18″E']
['Eastern', '東區', '574,500', '18.56', '31,217.67', 'Hong Kong Island', '22.28411 N 114.22414 E', '22°17′03″N', '114°13′27″E']
['Southern', '南區', '269,200', '38.85', '6,962.68', 'Hong Kong Island', '22.24725 N 114.15884 E', '22°14′50″N', '114°09′32″E']
['Wan Chai', '灣仔區', '150,900', '9.83', '15,300.10', 'Hong Kong Island', '22.27968 N 114.17168 E', '22°16′47″N', '114°10′18″E']
['Sham Shui Po', '深水埗區', '390,600', '9.35', '41,529.41', 'Kowloon', '22.33074 N 114.1622 E', '22°19′51″N', '114°09′44″E']
['Kowloon City', '九龍城區', '405,400', '10.02', '40,194.70', 'Kowloon', '22.3282 N 114.19155 E', '22°19′42″N', '114°11′30″E']
['Kwun Tong', '觀塘區', '641,100', '11.27', '56,779.05', 'Kowloon', '22.31326 N 114.22581 E', '22°18′48″N', '114°13′33″E']
['Wong Tai Sin', '黃大仙區', '426,200', '9.30', '45,645.16', 'Kowloon', '22.33353 N 114.19686 E', '22°20′01″N', '114°11′49″E']
['Yau Tsim Mong', '油尖旺區', '318,100', '6.99', '44,864.09', 'Kowloon', '22.32138 N 114.1726 E', '22°19′17″N', '114°10′21″E']
['Islands', '離島區', '146,900', '175.12', '825.14', 'New Territories', '22.26114 N 113.94608 E', '22°15′40″N', '113°56′46″E']
['Kwai Tsing', '葵青區', '507,100', '23.34', '21,503.86', 'New Territories', '22.35488 N 114.08401 E', '22°21′18″N', '114°05′02″E']
['North', '北區', '310,800', '136.61', '2,220.19', 'New Territories', '22.49471 N 114.13812 E', '22°29′41″N', '114°08′17″E']
['Sai Kung', '西貢區', '448,600', '129.65', '3,460.08', 'New Territories', '22.38143 N 114.27052 E', '22°22′53″N', '114°16′14″E']
['Sha Tin', '沙田區', '648,200', '68.71', '9,433.85', 'New Territories', '22.38715 N 114.19534 E', '22°23′14″N', '114°11′43″E']
['Tai Po', '大埔區', '307,100', '136.15', '2,220.35', 'New Territories', '22.45085 N 114.16422 E', '22°27′03″N', '114°09′51″E']
['Tsuen Wan', '荃灣區', '303,600', '61.71', '4,887.38', 'New Territories', '22.36281 N 114.12907 E', '22°21′46″N', '114°07′45″E']
['Tuen Mun', '屯門區', '495,900', '82.89', '5,889.38', 'New Territories', '22.39163 N 113.9770885 E', '22°23′30″N', '113°58′38″E']
['Yuen Long', '元朗區', '607,200', '138.46', '4,297.99', 'New Territories', '22.44559 N 114.02218 E', '22°26′44″N', '114°01′20″E']

如何从不同的Wikipedia页面抓取数据？

2 个答案: