如何从美味汤汤中获取特定的单词短语?

时间:2018-04-25 09:26:51

标签: python beautifulsoup screen-scraping

我已经使用BeautifulSoup对我的代码进行了排序,并提出了这个问题:

<bound method Tag.prettify of <script type="text/javascript">var LifeTimeStats = [{"Key":"Top 3","Value":"31"},{"Key":"Top 5s","Value":"36"},{"Key":"Top 3s","Value":"13"},{"Key":"Top 6s","Value":"27"},{"Key":"Top 12s","Value":"76"},{"Key":"Top 25s","Value":"58"},{"Key":"Score","Value":"99,788"},{"Key":"Matches Played","Value":"502"},{"Key":"Wins","Value":"9"},{"Key":"Win%","Value":"2%"},{"Key":"Kills","Value":"730"},{"Key":"K/d","Value":"1.48"}];</script>>

我正在尝试获取特定值“730” 从这个:

{"Key":"Kills","Value":"730"}

由于没有HTML标签,我可以排序。我不知道如何获得这个特定的价值。你有什么想法吗?

也许有另一种解决方案可以实现目标...... 这是完整的代码:

#----WEB INPUT BASIC----

    #import bs4
    from urllib.request import urlopen as uReq
    from urllib.request import Request, urlopen
    from  bs4 import BeautifulSoup as soup


    #setting my url
    url = 'https://fortnitetracker.com/profile/psn/Rehgum'

    #making my https page work
    req = Request(url, headers={'User-Agent': 'Mozilla/5.0'})

    web_byte = urlopen(req).read()
    webpage = web_byte.decode('utf-8')
    urlopen(req).close()

    #html parsing
    page_soup = soup(webpage, "html.parser")

    lifetime = page_soup.findAll("script",{"type":"text/javascript"})

    stats = lifetime[3]

    specific = stats.prettify

    value = specific.text

    #from here there is just code to put that value in a .txt file

2 个答案:

答案 0 :(得分:1)

这只是你能做什么的想法:

  1. 将JS代码解压缩为Python变量。
  2. 进行正则表达式操作,提取变量的值。
  3. “JSONify”这样的变量值。
  4. 提取您需要的数据。
  5. 作为摘录:

    a = '''var LifeTimeStats = [{"Key":"Top 3","Value":"31"},{"Key":"Top 5s","Value":"36"},{"Key":"Top 3s","Value":"13"},{"Key":"Top 6s","Value":"27"},{"Key":"Top 12s","Value":"76"},{"Key":"Top 25s","Value":"58"},{"Key":"Score","Value":"99,788"},{"Key":"Matches Played","Value":"502"},{"Key":"Wins","Value":"9"},{"Key":"Win%","Value":"2%"},{"Key":"Kills","Value":"730"},{"Key":"K/d","Value":"1.48"}];'''
    
    b = re.findall(r'var.*?=\s*(.*?);', a)[0]
    c = json.loads(b)
    

    请参阅我写的dummy full code

    <强>更新

    看到完整的代码后,This可以解决您的问题。

答案 1 :(得分:0)

我终于搞定了! 产生我的错误的是&#34; def loop():&#34;一部分。

这是最终的工作代码:

def loop():
    from urllib.request import Request, urlopen
    from  bs4 import BeautifulSoup as soup
    import json
    import re
    import time


    #setting my url
    url = 'https://fortnitetracker.com/profile/psn/Rehgum'

    #making my https page work
    req = Request(url, headers={'User-Agent': 'Mozilla/5.0'})

    web_byte = urlopen(req).read()
    webpage = web_byte.decode('utf-8')
    urlopen(req).close()

    #html parsing
    page_soup = soup(webpage, "html.parser")

    lifetime = page_soup.findAll("script",{"type":"text/javascript"})

    stats = lifetime[3]

    stats_var = re.findall(r'var.*?=\s*(.*?);', stats.text)[0]

    vals = json.loads(stats_var)

    for val in vals:
        if val['Key'] == 'Kills':
            num_kills = val['Value']
            break

    print('Num kills = {}'.format(num_kills))

    with open('lifetime_wins.txt', 'w') as fd:
        fd.write(str(num_kills))

    time.sleep(30)

    loop()


for i in range(1,2):
        loop()

while i<1:
    print ("Ende")

大&#34;谢谢你&#34;到@kazbeel。你救了我的一天! +代表