在BS4中刮擦巫妖的动作

时间:2018-09-04 18:51:01

标签: python selenium web-scraping beautifulsoup chess

我想创建一个关于国际象棋最佳动作的建议AI。 我真的没有在bs4中得到lichess的html标签。

在Chrome中,我实际上可以在bs4中看到Tag,它是基于脚本的命令。 我将显示一些代码,是为了更好地理解。

那是我用于抓取数据的脚本。

def scrape_data_from_lichess(url):
    response = get(url)
    html_soup = BeautifulSoup(response.text, 'html.parser')
    move_played = html_soup.find_all('move')
    print(move_played)

while not scrapeData:
    scrape_data_from_lichess(webpage + url)
    time.sleep(10)

这是我的Google Devtool对标记位置的响应。 确实很奇怪,因为Chrome扩展程序“ Web抓取工具”实际上可以检测到这些标签。

<div class="moves">
  <index class="">1</index>
  <move class="">d4</move>
  <move class="">d6</move>
  <index class="">2</index>
  <move class="active">Nf3</move>
  <move class="">Nf6</move>
</div>

最后一个是我的html文本的bs4结果,它是简短版本。我修改了。

LichessRound.boot({
    data: {"game":{"player":"white","opponent":{"color":"black","name":null,"ai":1},"san":"d4","san":"d6","san":"Nf3",{"san":"Nf6"}]},

这是长版。

LichessRound.boot({
data: {"game":{"id":"4PAVr8QI","variant":{"key":"standard","name":"Standard","short":"Std"},"speed":"correspondence","perf":"correspondence","rated":false,"initialFen":"rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1","fen":"rnbqkb1r/ppp1pppp/3p1n2/8/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - 2 3","player":"white","turns":4,"startedAtTurn":0,"source":"ai","status":{"id":20,"name":"started"},"createdAt":1536082009777,"lastMove":"g8f6","opening":{"eco":"A46","name":"Indian Game: Wade-Tartakower Defense","ply":4}},"clock":null,"correspondence":null,"player":{"color":"white","name":null,"user":{"id":"gamelinetv","username":"Gamelinetv","online":true,"perfs":{"correspondence":{"games":0,"rating":1500,"rd":350,"prog":0,"prov":true}}},"rating":1500,"provisional":true,"version":4,"spectator":true},"opponent":{"color":"black","name":null,"ai":1},"orientation":"white","url":{"socket":"/4PAVr8QI/white/socket","round":"/4PAVr8QI/white"},"pref":{"animationDuration":300,"coords":2,"replay":2,"clockTenths":1,"clockBar":true,"highlight":true,"destination":true,"rookCastle":true,"showCaptured":true},"evalPut":false,"steps":[{"ply":0,"uci":null,"san":null,"fen":"rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"},{"ply":1,"uci":"d2d4","san":"d4","fen":"rnbqkbnr/pppppppp/8/8/3P4/8/PPP1PPPP/RNBQKBNR b KQkq - 0 1"},{"ply":2,"uci":"d7d6","san":"d6","fen":"rnbqkbnr/ppp1pppp/3p4/8/3P4/8/PPP1PPPP/RNBQKBNR w KQkq - 0 2"},{"ply":3,"uci":"g1f3","san":"Nf3","fen":"rnbqkbnr/ppp1pppp/3p4/8/3P4/5N2/PPP1PPPP/RNBQKB1R b KQkq - 1 2"},{"ply":4,"uci":"g8f6","san":"Nf6","fen":"rnbqkb1r/ppp1pppp/3p1n2/8/3P4/5N2/PPP1PPPP/RNBQKB1R w KQkq - 2 3"}]},

我知道Selenium可以解决问题。但是它启动了一个新的浏览器,我只想粘贴脚本并获取移动数据。因此,我决定使用BS4,但我愿意接受新的想法。

希望你们能帮助我。

最佳问候 ren

0 个答案:

没有答案
相关问题