Question

我曾尝试使用beautifulsoup而不是使用Twitter API来抓取我的Twitter帐户。我收到一个错误，指出有效负载字典中的语法无效。我的代码是：

    import requests
from bs4 import BeautifulSoup

payload={
":authority":"twitter.com",
":method":"POST",
":path":"/sessions",
":scheme":"https",
"accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"accept-encoding":"gzip, deflate, br",
"accept-language":"en-US,en;q=0.9",
"cache-control":"max-age=0",
"content-length":"993",
"content-type":"application/x-www-form-urlencoded",
"cookie":"_twitter_sess=BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCKnkoiBhAToMY3NyZl9p%250AZCIlYjBmNmQ0NmY5NTVmMmVkNTgwYjM2NTU0YzQ2NDY4NzA6B2lkIiUwYTg3%250ANzU1MmU3ZmY3ZWE3NjAyYzIwZmU5ZjkxZWQ5OQ%253D%253D--43822fe18903b3578a1f5deaeb232a8186b75e7d;" "personalization_id"="v1_jQItTVsewWTebwVuukbh6g=="; "guest_id"="v1%3A151667100176659384"; "external_referer=padhuUp37zjgzgv1mFWxJzx4vPNPpMnZ52M34anUtp0%3D|0|8e8t2xd8A2w%3D"; "ct0"="6d7781aae304f30bb5d6d0d64fc34ffe"; "_ga"="GA1.2.1441535218.1516671002"; "_gid"="GA1.2.1585412472.1516671002"; "_gat"="1",
"origin":"https://twitter.com",
"referer":"https://twitter.com/login",
"upgrade-insecure-requests":"1",
"user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36",
"username":"aaaaaaa",
"password":"ppppppp",
"authenticity_token":"40a1cc3105efeb2f1a978cf97502bc4e8f97580b"

}


res = requests.get("https://twitter.com/login",data=payload)
soup = BeautifulSoup(res.text,"lxml")
print(soup.prettify())
for item in soup.find_all(class_="title"):
    print(item.text)

我收到错误说＆＃34;语法无效＆＃34; at＆＃34; cookie＆＃34;部分有效载荷。那里有什么错误？请帮帮我。

更新我保留了＆＃39;＆＃39;像这样的cookie部分。

"cookie":'"_twitter_sess"="BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCKnkoiBhAToMY3NyZl9p%250AZCIlYjBmNmQ0NmY5NTVmMmVkNTgwYjM2NTU0YzQ2NDY4NzA6B2lkIiUwYTg3%250ANzU1MmU3ZmY3ZWE3NjAyYzIwZmU5ZjkxZWQ5OQ%253D%253D--43822fe18903b3578a1f5deaeb232a8186b75e7d;" "personalization_id"="v1_jQItTVsewWTebwVuukbh6g=="; "guest_id"="v1%3A151667100176659384"; "external_referer=padhuUp37zjgzgv1mFWxJzx4vPNPpMnZ52M34anUtp0%3D|0|8e8t2xd8A2w%3D"; "ct0"="6d7781aae304f30bb5d6d0d64fc34ffe"; "_ga"="GA1.2.1441535218.1516671002"; "_gid"="GA1.2.1585412472.1516671002"; "_gat"="1"'

但问题是当我打印soup.prettify（）时，它显示为空屏幕。

Answer 1

您需要删除Cookie之间的引号

"_twitter_sess=BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCKnkoiBhAToMY3NyZl9p%250AZCIlYjBmNmQ0NmY5NTVmMmVkNTgwYjM2NTU0YzQ2NDY4NzA6B2lkIiUwYTg3%250ANzU1MmU3ZmY3ZWE3NjAyYzIwZmU5ZjkxZWQ5OQ%253D%253D--43822fe18903b3578a1f5deaeb232a8186b75e7d; personalization_id=v1_jQItTVsewWTebwVuukbh6g==; guest_id=v1%3A151667100176659384; external_referer=padhuUp37zjgzgv1mFWxJzx4vPNPpMnZ52M34anUtp0%3D|0|8e8t2xd8A2w%3D; ct0=6d7781aae304f30bb5d6d0d64fc34ffe; _ga=GA1.2.1441535218.1516671002; _gid=GA1.2.1585412472.1516671002;_gat=1"

在requests.get（）中获取错误

1 个答案: