为什么我的R代码可以在虚拟群集上正常运行,但不能在物理机上正常运行?

时间:2019-10-30 04:21:14

标签: r rstudio cluster-computing data-science anova

我只是对我的代码有一个简短的问题。我在运行RStudio的虚拟集群上运行的代码与在物理机上运行的代码之间存在一些差异。 为了重现ANOVA表,我们必须创建一个R-Markdown文件。我在集群上运行了我的代码。

这是我的代码:

```{r, message=FALSE, warning=FALSE}
wine <- read.csv("wine.csv")

cultivar <- as.factor( wine[, "Cultivar"])
alcohol <- wine[, "Alcohol"]

alcohol.list <- split(alcohol, cultivar)


alcohol.list

$`1`
 [1] 14.23 13.20 13.16 14.37 13.24 14.20 14.39 14.06 14.83 13.86 14.10 14.12 13.75 14.75 14.38 13.63 14.30 13.83 14.19 13.64
[21] 14.06 12.93 13.71 12.85 13.50 13.05 13.39 13.30 13.87 14.02 13.73 13.58 13.68 13.76 13.51 13.48 13.28 13.05 13.07 14.22
[41] 13.56 13.41 13.88 13.24 13.05 14.21 14.38 13.90 14.10 13.94 13.05 13.83 13.82 13.77 13.74 13.56 14.22 13.29 13.72

$`2`
 [1] 12.37 12.33 12.64 13.67 12.37 12.17 12.37 13.11 12.37 13.34 12.21 12.29 13.86 13.49 12.99 11.96 11.66 13.03 11.84 12.33
[21] 12.70 12.00 12.72 12.08 13.05 11.84 12.67 12.16 11.65 11.64 12.08 12.08 12.00 12.69 12.29 11.62 12.47 11.81 12.29 12.37
[41] 12.29 12.08 12.60 12.34 11.82 12.51 12.42 12.25 12.72 12.22 11.61 11.46 12.52 11.76 11.41 12.08 11.03 11.82 12.42 12.77
[61] 12.00 11.45 11.56 12.42 13.05 11.87 12.07 12.43 11.79 12.37 12.04

$`3`
 [1] 12.86 12.88 12.81 12.70 12.51 12.60 12.25 12.53 13.49 12.84 12.93 13.36 13.52 13.62 12.25 13.16 13.88 12.87 13.32 13.08
[21] 13.50 12.79 13.11 13.23 12.58 13.17 13.84 12.45 14.34 13.48 12.36 13.69 12.85 12.96 13.78 13.73 13.45 12.82 13.58 13.40
[41] 12.20 12.77 14.16 13.71 13.40 13.27 13.17 14.13

oneway <- function(z)
  {
   ni <- sapply(z, length)
   yi_bar <- sapply(z, mean)
   s2i <- sapply(z, sd)
   Y_bar <- mean(unlist(z))
   g <- length(z)
   N <-length(unlist(z))

   Within_SS = sum((ni-1) * s2i^2)
   Between_SS = sum(ni *((yi_bar)-(Y_bar))^2)

   DF_Within = (N - g)
   DF_Between = (g - 1)

   list("WithinSS" = Within_SS, "BetweenSS"= Between_SS, "DFWithin" = DF_Within, "DFBetween" = DF_Between)

 }


 alcohol.aov <- oneway(alcohol.list)

 alcohol.aov

oneway.table <- function(z)
 {
   Mean_SSW <- z[[1]]/z[[3]]
   Mean_SSB <- z[[2]]/z[[4]]
   F_value <- (Mean_SSB/Mean_SSW)
   P_value <- pf(F_value, DF_Between, DF_Within, lower.tail = FALSE)

   anova <- matrix(c( z[[4]], z[[3]], z[[2]], z[[1]], Mean_SSB, Mean_SSW, F_value, NA, P_value, NA), ncol =5)
   dimnames(anova) <- list("Group" = c("cultivar", "Residuals"), "ANOVA" = c("DF", "Sum_Sq", "Mean_Sq", "F_value", "P_value"))

   printCoefmat(anova, signif.stars = TRUE, has.Pvalue = TRUE, digits = 3, na.print="")

 }

 oneway.table(alcohol.aov)

我在虚拟集群上所做的代码工作得很好,并且能够重现此ANOVA表:

               DF  Sum_Sq Mean_Sq F_value P_value    
cultivar    2.000  70.795  35.397     135  <2e-16 ***
Residuals 175.000  45.859   0.262                    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

但是,当我在本地计算机上运行它时,却收到以下错误消息:

Error in pf(F_value, DF_Between, DF_Within, lower.tail = FALSE) : object 'DF_Between' not found

我了解我的DF_Between在第二个代码块中找不到,但是为什么它可以在群集中而不是在本地计算机上工作?

我也重新运行我的代码,这次将定义添加到变量中:

oneway.table <- function(z)
 {

   g <- length(z)
   N <-length(unlist(z))

   DF_Within <- (N - g)
   DF_Between <- (g - 1)

   Mean_SSW <- z[[1]]/z[[3]]
   Mean_SSB <- z[[2]]/z[[4]]
   F_value <- (Mean_SSB/Mean_SSW)
   P_value <- pf(F_value, DF_Between, DF_Within, lower.tail = FALSE)



   anova <- matrix(c( z[[4]], z[[3]], z[[2]], z[[1]], Mean_SSB, Mean_SSW, F_value, NA, P_value, NA), ncol =5) 
   dimnames(anova) <- list("Group" = c("cultivar", "Residuals"), "ANOVA" = c("DF", "Sum_Sq", "Mean_Sq", "F_value",           "P_value")) 

   printCoefmat(anova, signif.stars = TRUE, has.Pvalue = TRUE, digits = 3, na.print="")

 }

 oneway.table(alcohol.aov)

但是现在,我的输出看起来像这样:

       ANOVA
Group            DF  Sum_Sq Mean_Sq F_value P_value
  cultivar    2.000  70.795  35.397     135        
  Residuals 175.000  45.859   0.262                

没有明显的星星或任何P_Value,如果有人可以提供帮助,将不胜感激。

1 个答案:

答案 0 :(得分:0)

解决方案

这是无需说明即可解决的方法。

创建可复制的示例:

alcohol.list <- list("1"=c(14.2, 13.2), 
                     "2"=c(12.3, 12.3),
                     "3"=c(12.8, 12.9))
alcohol.list

您未曾动过的oneway功能:

oneway <- function(z)
  {
   ni <- sapply(z, length)
   yi_bar <- sapply(z, mean)
   s2i <- sapply(z, sd)
   Y_bar <- mean(unlist(z))
   g <- length(z)
   N <-length(unlist(z))

   Within_SS = sum((ni-1) * s2i^2)
   Between_SS = sum(ni *((yi_bar)-(Y_bar))^2)

   DF_Within = (N - g)
   DF_Between = (g - 1)

   list("WithinSS" = Within_SS, "BetweenSS"= Between_SS, "DFWithin" = DF_Within, "DFBetween" = DF_Between)

 }


 alcohol.aov <- oneway(alcohol.list)

最后,您的oneway.tablep.value

oneway.table <- function(z)
 {
   Mean_SSW <- z$WithinSS/z$DFWithin
   Mean_SSB <- z$BetweenSS/z$DFBetween
   F_value <- (Mean_SSB/Mean_SSW)
   P_value <- pf(F_value, z$DFBetween, z$DFWithin, lower.tail = FALSE)

   anova <- matrix(c(z[[4]], z[[3]], z[[2]], z[[1]], Mean_SSB, Mean_SSW, F_value, NA, P_value, NA), ncol =5)
   dimnames(anova) <- list("Group" = c("cultivar", "Residuals"), "ANOVA" = c("DF", "Sum_Sq", "Mean_Sq", "F_value", "P_value"))

   printCoefmat(anova, signif.stars = TRUE, has.Pvalue = TRUE, digits = 3, na.print="")

 }
oneway.table(alcohol.aov)

返回:

             DF Sum_Sq Mean_Sq F_value P_value  
cultivar  2.000  1.990   0.995    5.91   0.091 .
Residuals 3.000  0.505   0.168                  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

说明

在以下代码中,DF_Between不是在pf()方法调用之前创建的。实际上,DF_Within也没有创建,并且在该范围中不存在。

例如,这可以工作:

# create DF_Between and DF_Within first and pass in all three as arguments
oneway.table <- function(z, DF_Between, DF_Within){
   Mean_SSW <- z[[1]]/z[[3]]
   Mean_SSB <- z[[2]]/z[[4]]
   F_value <- (Mean_SSB/Mean_SSW)
   P_value <- pf(F_value, DF_Between, DF_Within, lower.tail = FALSE)

   ...
 }

这也可以工作:

oneway.table <- function(z){
   Mean_SSW <- z[[1]]/z[[3]]
   Mean_SSB <- z[[2]]/z[[4]]
   F_value <- (Mean_SSB/Mean_SSW)
   # provided that z is a list with the two elements
   P_value <- pf(F_value, z$DF_Between, z$DF_Within, lower.tail = FALSE)

   ...
 }

这也可以:

oneway.table <- function(z){
   Mean_SSW <- z[[1]]/z[[3]]
   Mean_SSB <- z[[2]]/z[[4]]
   F_value <- (Mean_SSB/Mean_SSW)
   # create DF_Between and DF_Within directly in here
   g <- length(z)
   N <-length(unlist(z))
   DF_Within <- (N - g)
   DF_Between <- (g - 1)
   P_value <- pf(F_value, DF_Between, DF_Within, lower.tail = FALSE)

   ...
 }

无论选择哪种方式,您都只需要了解R使用的词法作用域规则。为您省去冗长而乏味的解释,这就是它的过程:

  

search process that occurs如下:

     
      
  • 如果在以下环境中找不到符号的值:   函数已定义,然后在父级中继续搜索   环境。
  •   
  • 搜索继续按父级顺序进行   直到我们到达顶层环境为止;这通常是   全局环境(工作区)或包的名称空间。
  •   
  • 在顶级环境之后,搜索继续在搜索列表中向下进行   直到我们遇到了空旷的环境。
  •   

在本地计算机的环境中,它首先在定义该功能的环境DF_Between中搜索DF_Withinoneway.table。找不到它,因此在父环境中搜索DF_BetweenDF_Within,也没有找到它,并且命中了空环境。

但是在您的群集上,它首先在定义该功能的环境DF_Between中搜索DF_Withinoneway.table。没有找到它,因此在父环境中搜索DF_BetweenDF_Within并找到了它。因此没有引发错误或异常。

您可以通过运行ls()进行打印以确认并确认DF_WithinDF_Between是否存在于群集的父环境中,而不是在本地计算机上。