如何将PGN数据读入DataFrame

时间:2017-09-25 00:48:12

标签: r regex dataframe chess

我有一个大量国际象棋游戏的单一.pgn(便携式游戏记谱法)。游戏包含在文件中,如下所示:

[Event "FIDE World Cup 2017"]
[Site "Tbilisi GEO"]
[Date "2017.09.05"]
[Round "1.1"]
[White "Carlsen, Magnus"]
[Black "Balogun, Oluwafemi"]
[Result "1-0"]
[WhiteTitle "GM"]
[BlackTitle "FM"]
[WhiteElo "2822"]
[BlackElo "2255"]
[ECO "B00"]
[Opening "King's pawn opening"]
[WhiteFideId "1503014"]
[BlackFideId "8501246"]
[EventDate "2017.09.03"]

1. e4 d6 2. d4 g6 3. Bc4 Nf6 4. Qe2 Nc6 5. Nf3 Bg7 6. O-O Bg4 7. c3 O-O         
8. h3 Bxf3 9. Qxf3 e5 10. Rd1 Qe8 11. d5 Ne7 12. Qe2 Nh5 13. Bb5 Qc8 
14. Na3 a6 15. Ba4 f5 16. Bc2 f4 17. Qg4 Qxg4 18. hxg4 Nf6 19. g5 Nd7 
20. Nc4 b6 21. b4 h6 22. gxh6 Bxh6 23. g4 Nf6 24. f3 Bg5 25. Kg2 Kg7 
26. a4 Bh4 27. Bd2 g5 28. Rh1 Ng6 29. Kf1 Rh8 30. Ke2 Bg3 31. a5 b5 32. 
Na3 Ne7 33. c4 c6 34. dxc6 Nxc6 35. Bc3 Rxh1 36. Rxh1 bxc4 37. Nxc4 Rb8 
38. Nxd6 Kg6 39. Nf5 1-0

[Event "FIDE World Cup 2017"]    
etc...

我想创建一个包含此数据的数据框,其中列标题是每个字符串左侧的单词,数据是字符串。然后是PGN字符串的单独列。

我试图通过R: How to read in a PGN as a Data Frame的影响来尝试这一点并提出:

pgn <- read.table("~/Desktop/GitHub/Chess/test.pgn", quote="", 
stringsAsFactors=FALSE)

# get column names
column_names <- sub("\\[(\\w+).+", "\\1", pgn[1:17,1])
column_names[17] <- "PGN"
#create DF
pgn.df <- data.frame(matrix(sub("\\[\\w+ \\\"(.+)\\\"\\]", "\\1", 
                     pgn[,1]),byrow=TRUE, ncol=17))

names(pgn.df) <- column_names

这里的问题是我的pgn信息是多行。那么有没有办法在我的正则表达式中解释这一点?或者一种自动更改文件以使pgn成为单行的方法?

谢谢!

2 个答案:

答案 0 :(得分:1)

我仍然建议在预备步骤中使用(更新的)替换RegEx删除不需要的中断,如下所示:

/(?:[^\[\]\n\S])\s*\n/ /g

您可以在线测试here(使用PGN作为输入文本)。但是像我这样逃避R中的特殊字符时遇到了一些问题 因此我决定改用Perl。

use strict;
use File::Slurp;
my $text = read_file($ARGV[0]);
$text =~ s/(?:[^\[\]\n\S])\s*\n/ /g;
write_file($ARGV[0], $text);

这可以从R中调用

system("perl ~/Desktop/regex.pl ~/Desktop/test.pgn")

答案 1 :(得分:1)

我还没有在Windows或Linux上对此进行测试,但该软件包基于的C代码库声称非常便携。您需要一个R设置,可以从源代码进行编译(例如,如果您在Windows上,则需要Rtools)。

安装:

devtools::install_github("hrbrmstr/pigeon")

使用(tidyverse并不真正需要包工作,但IMO打印的数据帧比内置的基本R打印功能更干净):

library(pigeon)
library(tidyverse)

这是一个带有内置数据集的小测试,可能是您正在使用的数据集:

fide <- read_pgn(system.file("extdata", "r7.pgn", package="pigeon"))

fide
## # A tibble: 2 x 12
##            Event    Site       Date Round               White               Black  Result WhiteElo BlackElo   ECO
## *          <chr>   <chr>      <chr> <chr>               <chr>               <chr>   <chr>    <chr>    <chr> <chr>
## 1 World Cup 2017 Tbilisi 2017.09.23  44.1 Aronian Levon (ARM)    Ding Liren (CHN) 1/2-1/2     2799     2777   A18
## 2 World Cup 2017 Tbilisi 2017.09.24  45.1    Ding Liren (CHN) Aronian Levon (ARM) 1/2-1/2     2777     2799   E06
## # ... with 2 more variables: LiveChessVersion <chr>, Moves <list>

glimpse(fide)
## Observations: 2
## Variables: 12
## $ Event            <chr> "World Cup 2017", "World Cup 2017"
## $ Site             <chr> "Tbilisi", "Tbilisi"
## $ Date             <chr> "2017.09.23", "2017.09.24"
## $ Round            <chr> "44.1", "45.1"
## $ White            <chr> "Aronian Levon (ARM)", "Ding Liren (CHN)"
## $ Black            <chr> "Ding Liren (CHN)", "Aronian Levon (ARM)"
## $ Result           <chr> "1/2-1/2", "1/2-1/2"
## $ WhiteElo         <chr> "2799", "2777"
## $ BlackElo         <chr> "2777", "2799"
## $ ECO              <chr> "A18", "E06"
## $ LiveChessVersion <chr> "1.4.8", "1.4.8"
## $ Moves            <list> [c("c4", "Nf6", "Nc3", "e6", "e4", "d5", "cxd5", "exd5", "e5", "Ne4", "Nf3", "Bf5", "Be2"...

这是一个更大的测试:

tf <- tempfile(fileext = ".zip")
td <- tempdir()
download.file("https://www.pgnmentor.com/players/Adams.zip",  tf)
fil <- unzip(tf, exdir = td)

adams <- read_pgn(fil)

adams
## # A tibble: 2,982 x 11
##             Event      Site       Date Round              White              Black  Result WhiteElo BlackElo   ECO
##  *          <chr>     <chr>      <chr> <chr>              <chr>              <chr>   <chr>    <chr>    <chr> <chr>
##  1 Lloyds Bank op    London 1984.??.??     1     Adams, Michael    Sedgwick, David     1-0                     C05
##  2 Lloyds Bank op    London 1984.??.??     3     Adams, Michael  Dickenson, Neil F     1-0              2230   C07
##  3 Lloyds Bank op    London 1984.??.??     4       Hebden, Mark     Adams, Michael     1-0     2480            B10
##  4 Lloyds Bank op    London 1984.??.??     5    Pasman, Michael     Adams, Michael     0-1     2310            D42
##  5 Lloyds Bank op    London 1984.??.??     6     Adams, Michael   Levitt, Jonathan 1/2-1/2              2370   B99
##  6 Lloyds Bank op    London 1984.??.??     9     Adams, Michael Saeed, Saeed Ahmed     1-0              2430   B56
##  7         BCF-ch Edinburgh 1985.??.??     1     Adams, Michael   Singh, Sukh Dave 1/2-1/2     2360     2080   B70
##  8         BCF-ch Edinburgh 1985.??.??     2 Abayasekera, Roger     Adams, Michael     1-0     2200     2360   B13
##  9         BCF-ch Edinburgh 1985.??.??     3     Adams, Michael    Jackson, Sheila 1/2-1/2     2360     2225   C85
## 10         BCF-ch Edinburgh 1985.??.??     4     Muir, Andrew J     Adams, Michael 1/2-1/2     2285     2360   E45
## # ... with 2,972 more rows, and 1 more variables: Moves <list>

glimpse(adams)
## Observations: 2,982
## Variables: 11
## $ Event    <chr> "Lloyds Bank op", "Lloyds Bank op", "Lloyds Bank op", "Lloyds Bank op", "Lloyds Bank op", "Lloyds ...
## $ Site     <chr> "London", "London", "London", "London", "London", "London", "Edinburgh", "Edinburgh", "Edinburgh",...
## $ Date     <chr> "1984.??.??", "1984.??.??", "1984.??.??", "1984.??.??", "1984.??.??", "1984.??.??", "1985.??.??", ...
## $ Round    <chr> "1", "3", "4", "5", "6", "9", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "?", "1", "...
## $ White    <chr> "Adams, Michael", "Adams, Michael", "Hebden, Mark", "Pasman, Michael", "Adams, Michael", "Adams, M...
## $ Black    <chr> "Sedgwick, David", "Dickenson, Neil F", "Adams, Michael", "Adams, Michael", "Levitt, Jonathan", "S...
## $ Result   <chr> "1-0", "1-0", "1-0", "0-1", "1/2-1/2", "1-0", "1/2-1/2", "1-0", "1/2-1/2", "1/2-1/2", "1-0", "1/2-...
## $ WhiteElo <chr> "", "", "2480", "2310", "", "", "2360", "2200", "2360", "2285", "2360", "2250", "2360", "2225", "2...
## $ BlackElo <chr> "", "2230", "", "", "2370", "2430", "2080", "2360", "2225", "2360", "2245", "2360", "2260", "2360"...
## $ ECO      <chr> "C05", "C07", "B10", "D42", "B99", "B56", "B70", "B13", "C85", "E45", "C84", "B10", "C85", "A22", ...
## $ Moves    <list> [c("e4", "e6", "d4", "d5", "Nd2", "Nf6", "e5", "Nfd7", "f4", "c5", "c3", "Nc6", "Ndf3", "cxd4", "...

使用一个完善的C“库”(它在技术上不是一个库,但我把它变成一个库)的一个好处是,它不仅仅是模式匹配。如果游戏文件格式错误,则无法正确解析(因为它不应该)。

我需要通过ASAN / UBSAN / Valgrind运行它以确保没有内存泄漏,但如果这最终有用,请告诉我,我将完成pkg上的角落。