未找到'itms-appss://itunes.apple.com/us/app/fiverr-freelance-services/id346080608?mt = 8&ign-mpt = uo%3D4'的连接适配器
我正在从Apple App Store抓取应用程序信息。这几分钟一直持续到抓取。
with open('App_Store_Links.csv', newline='') as f_urls, open('appsinfo.csv', 'w', newline='') as f_output:
csv_urls = csv.reader(f_urls)
csv_output = csv.writer(f_output)
csv_output.writerow(['App Name', 'Category','Size','Developer','Age Rating','Rating','Rating Numbers'])
headers = requests.utils.default_headers()
headers['User-Agent'] = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36'
for line in csv_urls:
r = requests.get(line[0].strip(),headers=headers).text
soup = BeautifulSoup(r, 'lxml')
for i, x in enumerate(line):
x = x.replace("itms-appss","https")
line[i] = x
try:
app_name = soup.find('h1', class_='product-header__title app-header__title').text
Category = soup.findAll('dd', class_='information-list__item__definition l-column medium-9 large-6')[2].text
Size = soup.findAll('dd', class_='information-list__item__definition l-column medium-9 large-6')[1].text
Developer = soup.findAll('dd', class_='information-list__item__definition l-column medium-9 large-6')[0].text
Age_Rating = soup.find('span', class_='badge badge--product-title').text
Price = soup.find('li', class_='inline-list__item inline-list__item--bulleted app-header__list__item--price').text
Rating_number = soup.find('div', class_='we-customer-ratings__count small-hide medium-show').text
Rating = soup.find('span', class_='we-customer-ratings__averages__display').text
except Exception:
pass
csv_output.writerow([app_name,Category,Size,Developer,Age_Rating,Rating,Rating_number])
然后我在运行代码时收到此错误。 p.s App_Store_Links.csv内部的链接都没有任何itms-appss网址,因此我不确定发生了什么
No connection adapters were found for 'itms-appss://itunes.apple.com/us/app/fiverr-freelance-services/id346080608?mt=8&ign-mpt=uo%3D4'
答案 0 :(得分:0)
通过在请求代码中将allow_redirects设置为false来解决
r = requests.get(line[0].strip(),allow_redirects=False,headers=headers).text