我有一个页面,需要登录到网站废品价格
因此,我有一个带有解析方法的类,该类在成功登录后会调用。
但是我只得到没有价格的页面,所以我得到NoneType对象不可下标
class Parser(object):
ses = requests.Session()
def parse(self, urls):
url = urls[1]
try:
r = self.ses.get(url)
sleep(3)
if r.status_code == 200:
Myhtml = r.text
soup = BeautifulSoup(Myhtml, 'html.parser')
print('Get Soup..')
price = soup.find('span', attrs={'itemprop': 'price'})
availability = soup.select_one('div.availability > span')['class'][0]
#Checking Conditions
data = [urls[0], price_text, availability_text]
return data
else:
print(r.status_code.text)
except TypeError as e:
print(e)
return []
except IndexError as e:
print(e)
return []
except AttributeError as e:
print(e)
return []
except Exception as e:
print(str(e))
return []
if __name__ == "__main__":
print('Starting..')
browser = webdriver.Chrome('C:\\chromedriver.exe')
browser.get('https://www.acihellas.gr/login')
#LOGIN PROCESS Passing Username and Password into form
s = requests.Session()
parser = Parser()
parser.ses = s
browser.quit()
for link in web_links:
parser.parse(link)
#export to csv
我不知道这是否有用,但是在
r = self.ses.get(url)
ses有两个cookie,而r具有一个cookie
我在这里做错了什么?我尝试了
cookies = browser.get_cookies()
for cookie in cookies:
s.cookies.set(cookie['name'], cookie['value'])
但同样的结果 如何保留我要解析的每个URL的身份验证cookie?