Question

我的python脚本，使用BeautifulSoup，似乎无法掌握页面上div的字样，是否有特定原因？我可以抓取个人资料图片来计算消息数量，但不是文本本身。

（供参考，我使用过这个页面：http://whoscall.in/1/2392247496/）

if(website == "1"):  
  reqInput = "http://whoscall.in/1/%s/" % (teleWho)
  urlfile = urllib2.Request(reqInput)
  print (reqInput)
  time.sleep(1)
  requestRec = requests.get(reqInput)
  soup = BeautifulSoup(requestRec.content, "lxml")
  noMatch = soup.find(text=re.compile(r"no reports yet on the phone number"))
  print(requestRec.content)# #only if needed#
  type(noMatch) is str
  if noMatch is None:
     worksheet.write(idx+1, 2, "Got a hit")
     howMany = soup.find_all('img',{'src':'/default-avatar.gif'})
     howManyAreThere = len(howMany)
     worksheet.write(idx+1,1,howManyAreThere)
     print (howManyAreThere)
     scamNum = soup.find_all(text=("scam"),recursive=True)
     #,'scam','Scammer','scammer'#
     scamCount = len(scamNum)
     print(scamNum)
     searchTerms = {scamCount:scamCount}
     sentiment = max(searchTerms, key=searchTerms.get)
     worksheet.write(idx+1,3,sentiment)

我似乎无法从网页上删除文字“骗局”

我不确定为什么它拒绝找到该文本，因为其他美丽的汤代码完美无缺。

https://github.com/GarnetSunset/Haircuttery/

Answer 1

更改此行：

scamNum = soup.find_all(text=("scam"),recursive=True)

到：

scamNum = [ div.text for div in soup.find_all('div', {'style':'font-size:14px; margin:10px; overflow:hidden'}) if 'scam' in div.text.lower() ]

尝试使用多个单词：

words = [ 'word1', 'word2', ... ]
scamNum = [ div.text for div in soup.find_all('div', {'style':'font-size:14px; margin:10px; overflow:hidden'}) if any( word for word in words if word in div.text.lower()) ]

使用BeautifulSoup 4刮擦whoscall.in上的问题

1 个答案: