Question

我正试图从网页中获取与此餐厅L'atelier de willy相关的网站链接，但我做不到。

这就是在该页面中的显示方式（在同一位置，餐厅名称以粗体显示）：

我尝试过：

import requests
from bs4 import BeautifulSoup

link = "https://www.tripadvisor.fr/Restaurant_Review-g188644-d14788983-Reviews-Mozart_More_Than_Just_Ribs-Brussels.html"

res = requests.get(link,headers={"User-Agent":"Mozilla/5.0"})
soup = BeautifulSoup(res.text,"lxml")
website = soup.select_one("[class*='website']").get("data-ahref")
print(website)

我得到的输出：

q5aizCJEIWEVtIiYHVLaizCJEIWHEpttVcL4pIaQtipEnV1zS0pIaQaVMSpa1EVTVEEJc

我希望得到的东西：

https://mozart-resto.be/

如何使用请求取消该网站链接？

Answer 1

该网站使用的是“ asdf”编码器（我不确定这是否是官方名称）。但是使用Converting JavaScript code to Python的回复，您应该可以解码以下字符串：

d = "q5aizCJEIWEVtIiYHVLaizCJEIWHEpttVcL4pIaQtipEnV1zS0pIaQaVMSpa1EVTVEEJc"

def asdf(d):

  b = ""
  h = {
    "": ["&", "=", "p", "6", "?", "H", "%", "B", ".com", "k", "9", ".html", "n", "M", "r", "www.", "h", "b", "t", "a", "0", "/", "d", "O", "j", "http://", "_", "L", "i", "f", "1", "e", "-", "2", ".", "N", "m", "A", "l", "4", "R", "C", "y", "S", "o", "+", "7", "I", "3", "c", "5", "u", 0, "T", "v", "s", "w", "8", "P", 0, "g", 0],
    "q": [0, "__3F__", 0, "Photos", 0, "https://", ".edu", "*", "Y", ">", 0, 0, 0, 0, 0, 0, "`", "__2D__", "X", "<", "slot", 0, "ShowUrl", "Owners", 0, "[", "q", 0, "MemberProfile", 0, "ShowUserReviews", '"', "Hotel", 0, 0, "Expedia", "Vacation", "Discount", 0, "UserReview", "Thumbnail", 0, "__2F__", "Inspiration", "V", "Map", ":", "@", 0, "F", "help", 0, 0, "Rental", 0, "Picture", 0, 0, 0, "hotels", 0, "ftp://"],
    "x": [0, 0, "J", 0, 0, "Z", 0, 0, 0, ";", 0, "Text", 0, "(", "x", "GenericAds", "U", 0, "careers", 0, 0, 0, "D", 0, "members", "Search", 0, 0, 0, "Post", 0, 0, 0, "Q", 0, "$", 0, "K", 0, "W", 0, "Reviews", 0, ",", "__2E__", 0, 0, 0, 0, 0, 0, 0, "{", "}", 0, "Cheap", ")", 0, 0, 0, "#", ".org"],
    "z": [0, "Hotels", 0, 0, "Icon", 0, 0, 0, 0, ".net", 0, 0, "z", 0, 0, "pages", 0, "geo", 0, 0, 0, "cnt", "~", 0, 0, "]", "|", 0, "tripadvisor", "Images", "BookingBuddy", 0, "Commerce", 0, 0, "partnerKey", 0, "area", 0, "Deals", "from", "//", 0, "urlKey", 0, "'", 0, "WeatherUnderground", 0, "MemberSign", "Maps", 0, "matchID", "Packages", "E", "Amenities", "Travel", ".htm", 0, "!", "^", "G"]
  }

  #for a in range(len(d)):     ## REMOVE and Change this to a WHILE loop (see below)
  a = 0                        #  Manually initialize your loop
  while a < len(d):
    j = d[a]
    f = j
    list = []

    for key in h:
      list.append(key)

    if (j in list) and (a < len(d)):
       a = a + 1                  ## CANNOT DO THIS, if you a use "for a in.." loop.  So, we'll use a WHILE loop instead
       f = f + d[a]
    else:
       j = ""

    g = getOffset(ord(d[a]))
    if  g < 0 :                   ## REMEMBER TO UPDATE THIS LINE to REMOVE # or type(h[j][g]) is str:
        b = b + f
    else:
        b = b + str(h[j][g])
        #print b                  # REMOVE this line

    a = a + 1                     # Manually increment your WHILE loop
  return b

def getOffset(a):
    if(a >= 97 and a <= 122):
        return(a-61)
    if(a >= 65 and a <= 90):
        return(a-55)
    if(a >= 48 and a <=71):
        return(a-48)
    print ("\n\nERROR\n\n")
    return(-1)

print(asdf(d))

打印：

https://mozart-resto.be/mozart-brussel/?utm_source=tripadvisor&utm_medium=referral

编辑（用于选择链接）：

from bs4 import BeautifulSoup
import requests

url = 'https://www.tripadvisor.fr/Restaurant_Review-g188644-d14788983-Reviews-Mozart_More_Than_Just_Ribs-Brussels.html'

soup = BeautifulSoup(requests.get(url).text, 'lxml')

s = soup.select_one('div[data-ahref]:has(span:contains("Site Web"))')
if s:
    site_web = asdf(s['data-ahref']) # this is the decoder function
    print(site_web)

打印：

https://mozart-resto.be/mozart-brussel/?utm_source=tripadvisor&utm_medium=referral

Answer 2

~~很遗憾，我还没有评论，这应该是评论：（~~

因此，您的代码正在寻找data_ahref，它应该能得到您的输出显示的内容。网站的来源显示

<div class="is-hidden-mobile blEntry website  ui_link" data-ahref="q5aizCJEIWEVtIiYHVLaizCJEIWHEpttVcL4pIaQtipEnV1zS0pIaQaVMSpa1EVTVEEJc" data-column="2" data-trackingkey="URL_EATERY" data-eventname="bl_contact_website" data-blcontact="URL_HOTEL" onclick="widgetEvCall('handlers.onWebLinkClicked', event, this)"><span class="primary_icon ui_icon laptop"></span><span class="detail ">Site Web</span></div>

并说data-ahref="q5aizCJEIWEVtIiYHVLaizCJEIWHEpttVcL4pIaQtipEnV1zS0pIaQaVMSpa1EVTVEEJc"。这意味着您的代码（和bs4）正常工作。

另一件事是，当我单击Site Web链接时，我被带到this link，而不是您想要的链接。并且所需的链接在源代码中找不到。

那么，您确定要寻找正确的东西吗？

编辑： 看着Andrej Kesely的答案，我意识到答案的第一部分并不重要。我没有意识到data-ahref实际上是作为URL的编码字符串。

Answer 3

您为什么不签出https://html.python-requests.org/，其目的是为了使用它来解析网页。

使用请求无法抓取网站链接

3 个答案: