我无法使用python从Betfair网站解析XML

时间:2019-04-28 07:39:13

标签: xml python-3.x parsing web-scraping

我需要使用python从betfair网站获取一些数据 我需要一个链接: https://www.betfair.com/www/sports/exchange/readonly/v1/bymarket?_ak=nzIFcwyWhrlwYMrh&currencyCode=EUR&locale=ru&marketIds=1.158145690&rollupLimit=10&rollupModel=STAKE&types=MARKET_STATE,MARKET_RATES,MARKET_DESCRIPTION,EVENT,RUNNER_DESCRIPTION,RUNNER_STATE,RUNNER_EXCHANGE_PRICES_BEST,RUNNER_METADATA,MARKET_LICENCE

为解决此问题,我将打开的页面另存为xml文档,并使用此代码成功解析了该页面

from bs4 import BeautifulSoup as bs

filedata = open('bymarket.xml')

cont = bs(filedata, 'lxml')

course = cont.find('venue')
print(course.text)
container = cont.find('runners')
item = container.find_all('runnernode')
for horse in item:
    runner = horse.find('runnername').text
    odds = horse.find('availabletolay').find('price').text
    print(runner, odds)

使用此代码,我可以获得所需的结果。 但是,当我尝试直接从页面获取此数据时,没有任何反应- 找不到任何项目,但我没有。 请帮助非专业人士获取我需要的信息

1 个答案:

答案 0 :(得分:0)

您需要将Accept标头添加到请求中,并指定xml解析器

 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',

Py:

import requests
from bs4 import BeautifulSoup as bs

headers = {
     'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
    'User-Agent': 'Mozilla/5.0'
}
r = requests.get('https://www.betfair.com/www/sports/exchange/readonly/v1/bymarket?_ak=nzIFcwyWhrlwYMrh&currencyCode=EUR&locale=ru&marketIds=1.158145690&rollupLimit=10&rollupModel=STAKE&types=MARKET_STATE,MARKET_RATES,MARKET_DESCRIPTION,EVENT,RUNNER_DESCRIPTION,RUNNER_STATE,RUNNER_EXCHANGE_PRICES_BEST,RUNNER_METADATA,MARKET_LICENCE', headers = headers)
soup = bs(r.content, 'xml')
for horse in soup.select('RunnerNode'):
    name = horse.select_one('runnerName').text
    first_availableToLay_price_size =  horse.select_one('price').text
    print(name, first_availableToLay_price_size)

输出:

enter image description here