使用R从存储库导入.dat文件

时间:2018-05-11 18:42:17

标签: r dataframe import read.fwf

我想将此数据集导入df。我正在尝试使用read.fwf方法

将此SAS支持文件转换为R代码

按SAS文件

中的描述定义变量名称和长度
vars <- c('SEQN',   'HYK1A',    'HYK1B',    'HYK2A',    'HYK2B',    'HYK3CG',   'HYK3DG',   'HYK6SG',   'HYK8SG',   'HYK3CH',   'HYK3DH',   'HYK6SH',   'HYK8SH',   'HYK3CI',   'HYK3DI',   'HYK6SI',   'HYK8SI',   'HYK3CJ',   'HYK3DJ',   'HYK6SJ',   'HYK8SJ',   'HYK3CK',   'HYK3DK',   'HYK6SK',   'HYK8SK',   'HYK3CL',   'HYK3DL',   'HYK6SL',   'HYK8SL',   'HYK3CM',   'HYK3DM',   'HYK6SM',   'HYK8SM',   'HYK3CN',   'HYK3DN',   'HYK6SN',   'HYK8SN',   'HYK3CO',   'HYK3DO',   'HYK6SO',   'HYK8SO',   'HYK3CP',   'HYK3DP',   'HYK6SP',   'HYK8SP',   'HYK9DG',   'HYK9EG',   'HYK9FG',   'HYK11AG',  'HYK12SG',  'HYK9DH',   'HYK9EH',   'HYK9FH',   'HYK11AH',  'HYK12SH',  'HYK9DI',   'HYK9EI',   'HYK9FI',   'HYK11AI',  'HYK12SI',  'HYK9DJ',   'HYK9EJ',   'HYK9FJ',   'HYK11AJ',  'HYK12SJ',  'HYK9DK',   'HYK9EK',   'HYK9FK',   'HYK11AK',  'HYK12SK',  'HYK9DL',   'HYK9EL',   'HYK9FL',   'HYK11AL',  'HYK12SL',  'HYK9DM',   'HYK9EM',   'HYK9FM',   'HYK11AM',  'HYK12SM',  'HYK9DN',   'HYK9EN',   'HYK9FN',   'HYK11AN',  'HYK12SN',  'HYK9DO',   'HYK9EO',   'HYK9FO',   'HYK11AO',  'HYK12SO')
len <-c(7,  3,  3,  3,  3,  3,  3,  4,  4,  3,  3,  4,  4,  3,  3,  4,  4,  3,  3,  4,  4,  3,  3,  4,  4,  3,  3,  4,  4,  3,  3,  4,  4,  3,  3,  4,  4,  3,  3,  4,  4,  3,  3,  4,  4,  4,  4,  4,  6,  4,  4,  4,  4,  6,  4,  4,  4,  4,  6,  4,  4,  4,  4,  6,  4,  4,  4,  4,  6,  4,  4,  4,  4,  6,  4,  4,  4,  4,  6,  4,  4,  4,  4,  6,  4,  4,  4,  4,  6,  4)

从网络上检索DF

df <- read.fwf(".../you.dat",
                    widths = len, header = FALSE, n=10, stringsAsFactors = TRUE)
names(df) <- vars

可视化DF

head(df)

老实说,我不相信这个DF。我收到了太多NAs

@ 42-更新答案后更新。更快的方式

我使用SAScii库轻松改进了我的代码并且它可以正常工作。 但是,我发现更快的系统费用here

library(readr)
library(data.table)

#Parse SAS file
vars <- parse.SAScii(".../you.sas")

setDT(vars) #convert to data.table

#read to data frame
huge.df <- read_fwf(".../you.dat", 
                     fwf_widths(dput(vars[,width]),
                                col_names=(dput(vars[,varname]))),
                     progress = interactive()
)

2 个答案:

答案 0 :(得分:3)

几年前,我也尝试过这样做,并最终拼凑出一种方法,但与此同时,@ AnthonyDamico编写了SAScii - 包,所有这些努力现在都没有用了。感谢Anthony,感谢您为R用户提供公共使用数据所做的出色工作。我怀疑你在就业和咨询生涯中为凯撒基金会节省了数十万美元。

library(SAScii)
youth.tf <- tempfile()
daturl <- "https://wwwn.cdc.gov/nchs/data/nhanes3/2a/youthk.dat"
code_url ="https://wwwn.cdc.gov/nchs/data/nhanes3/2a/youthk.sas"
Sas_code <- url(code_url)
writeLines ( readLines(Sas_code) , con = youth.tf )
youth.fwf.parameters <- parse.SAScii( youth.tf , beginline = 5 )
str( youth.fwf.parameters )
#-----
'data.frame':   90 obs. of  4 variables:
 $ varname: chr  "SEQN" "HYK1A" "HYK1B" "HYK2A" ...
 $ width  : num  5 1 1 2 2 2 2 4 4 2 ...
 $ char   : logi  FALSE FALSE FALSE FALSE FALSE FALSE ...
 $ divisor: num  1 1 1 1 1 1 1 1 1 1 ...
#------

daturl <- "https://wwwn.cdc.gov/nchs/data/nhanes3/2a/youthk.dat"
in.youth <- read.fwf(daturl, widths=youth.fwf.parameters$width,
                       col.names= youth.fwf.parameters$varname)

我担心会有相当多的NA,并且您的代码似乎主要是成功。

str(in.youth)
'data.frame':   13944 obs. of  90 variables:
 $ SEQN   : int  7 12 13 14 16 20 21 23 26 27 ...
 $ HYK1A  : int  1 1 2 2 2 2 1 2 2 2 ...
 $ HYK1B  : int  2 2 2 2 2 2 2 2 2 2 ...
 $ HYK2A  : int  1 1 NA NA NA NA 1 NA NA NA ...
 $ HYK2B  : int  NA NA NA NA NA NA NA NA NA NA ...
 $ HYK3CG : int  1 3 NA NA NA NA 4 NA NA NA ...
 $ HYK3DG : int  2 3 NA NA NA NA 2 NA NA NA ...
 $ HYK6SG : int  30 30 NA NA NA NA 30 NA NA NA ...
 $ HYK8SG : int  24 1 NA NA NA NA 24 NA NA NA ...
 $ HYK3CH : int  NA NA NA NA NA NA NA NA NA NA ...
 $ HYK3DH : int  NA NA NA NA NA NA NA NA NA NA ...
 $ HYK6SH : int  NA NA NA NA NA NA NA NA NA NA ...
 $ HYK8SH : int  NA NA NA NA NA NA NA NA NA NA ...
 $ HYK3CI : int  NA NA NA NA NA NA NA NA NA NA ...
 $ HYK3DI : int  NA NA NA NA NA NA NA NA NA NA ...
 $ HYK6SI : int  NA NA NA NA NA NA NA NA NA NA ...
.... snipped the rest of the 90 variables

我只看到这些数据可能包含无效数据且似乎未正确加载:

 HYK11AG: Factor w/ 179 levels "      ","003.9 ",
HYK11AH: Factor w/ 108 levels "      ","011.9 ",
HYK11AK: Factor w/ 12 levels "      ","079.9 ",

这些似乎是ICD-9-CM代码,应该用字符格式读入R,但CDC SAS代码中的INPUT语句中没有正确的$条目。您可以使用“read.SAScii”函数获得一整套诊断功能:

in2 <- read.SAScii( daturl, code_url)

您可以根据代码簿中报告的值检查非NA值的计数:https://wwwn.cdc.gov/nchs/data/nhanes3/1a/YOUTH-acc.pdf

这是该代码簿第210页的部分屏幕截图,显示该页面上的三个“HYK__”变量应该有所有NA:

enter image description here

注意添加娱乐。这实际上现在是这个R财富的反例。该软件包中有很多项目引用SAS(通常不太完全有利),在我找到的那个软件之前我花了7到8次尝试:

fortunes::fortune("SAS")

Jim Gustafsson: I would like to put my SAS-code into R. Could I do that,
if yes, how?
Frank Harrell: Just reverse the procedure you use when you put R code into
SAS. ;)
   -- Jim Gustafsson and Frank Harrell
      R-help (February 2004)

答案 1 :(得分:0)

使用SAScii包,可以很容易地以通用格式 read.SAScii(数据集,sascode,zip = F)的单行代码执行。例如:

mydata <- read.SAScii (".../you.dat", ".../you.sas", zipped = F)