为什么我无法使用Selenium python抓取特定的Amazon音乐页面?

时间:2018-11-16 03:06:05

标签: python beautifulsoup amazon

https://www.amazon.com/Prettymuch-EP-PRETTYMUCH/dp/B07CF6YXDP

上面提到的纯粹的链接,而不是一次使用堆栈溢出标记单击该链接所指向的链接。

这是网址。

def get_soup(url):
headers = {'User-Agent':
           'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36',
           }
r = requests.get(url, headers=headers)
r.raise_for_status()
return BeautifulSoup(r.text, 'lxml')

url = input("Please enter an Amazon music url:")
soup = get_soup(url)

通过请求时出现错误,为什么会这样?

Please enter an Amazon music url:https://www.amazon.com/Prettymuch-EP- 
PRETTYMUCH/dp/B07CF6YXDP 
Traceback (most recent call last):
  File "D:/Pycharm (4)/selemin.py", line 4, in <module>
    import amazon
  File "D:\Pycharm (4)\amazon.py", line 63, in <module>
    soup = get_soup(url)
  File "D:\Pycharm (4)\amazon.py", line 12, in get_soup
    r.raise_for_status()
  File "C:\Users\HP\AppData\Local\Programs\Python\Python37-32\lib\site- 
    packages\requests\models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
 requests.exceptions.HTTPError: 404 Client Error: Not Found for url: 
 https://www.amazon.com/Prettymuch-EP-PRETTYMUCH/dp/B07CF6YXDP%20

1 个答案:

答案 0 :(得分:1)

看看错误告诉您什么-这是一个不同的URL。特别是,它以%20结尾,因此会引发错误。这意味着输入末尾有一个空格。我建议处理您的输入来避免这种情况,例如

new_url = url.strip()