传递mclapply()参数from(i in range)

时间:2016-08-08 15:36:47

标签: r parallel-processing mclapply

我试图这样做:

nmf.sub <- function(n){
sub.data.matrix <- data.matrix[, (index[n, ])] ## the index is a permutation of the original matrix at a 0.8 resampling proportion (doesn't really matter)
temp.result <- nmf(sub.data.matrix, rank = 2, seed = 12345) ## want to change 2 to i
return(temp.result)
}

class.list <- list()
for (i in nmf.rank){ ## nmf.rank is 2:4
results.list <- mclapply(mc.cores = 16, 1:resamp.iterations, function(n) nmf.sub(n)) ## resamp.iterations is 10, nmf.sub is defined above
}

但是,对于temp.result而言,不是在nmf中得到rank = 2,我想得到rank = i

知道我怎么能传递那个参数吗?只是通过mclapply传递它作为函数(n,i)不起作用。

1 个答案:

答案 0 :(得分:0)

你似乎有两个循环:一个用于nmf.rank中的i,另一个用于1:resamp.iterations中的n。因此,您需要将in都传递给nmf.sub,例如喜欢在:

nmf.sub <- function(n, i){
    ## the index is a permutation of the original matrix at a 0.8
    ## resampling proportion (doesn't really matter)
    sub.data.matrix <- data.matrix[, (index[n, ])] 
    ## want to change 2 to i
    temp.result <- nmf(sub.data.matrix, rank = i, seed = 12345)
    return(temp.result)
}


resamp.iterations <- 10
nmf.rank <- 2:4

res <- lapply(nmf.rank, function(i){
    results.list <- mclapply(mc.cores = 16, 1:resamp.iterations,
                             function(n) nmf.sub(n,i))
})
## then you can flatten/reshape res

关于效率的评论(下面):大部分数值计算是在nmf()函数中执行的,因此在每个进程/核心获得数字密集型工作的意义上,正确设置了循环。但是,为了加快计算速度,您可以考虑使用先前计算的结果,而不是种子12345(除非出于与您的问题相关的某些原因使用后一种子是必需的)。在以下示例中,我的执行时间减少了30-40%:

library(NMF)
RNGkind("L'Ecuyer-CMRG") ## always use this when using mclapply()
nr <- 19
nc <- 2e2
set.seed(123)
data.matrix <- matrix(rexp(nc*nr),nr,nc)

resamp.iterations <- 10
nmf.rank <- 2:4

index <- t(sapply(1:resamp.iterations, function(n) sample.int(nc,nc*0.8)))


nmf.sub <- function(n, i){
    sub.data.matrix <- data.matrix[ ,index[n, ]] 
    temp.result <- nmf(sub.data.matrix, rank = i, seed = 12345)
    return(temp.result)
}

## version 1
system.time({
    res <- lapply(nmf.rank, function(i){
        results.list <- mclapply(mc.cores = 16, 1:resamp.iterations,
                                 function(n) nmf.sub(n,i))
    })
})

## version 2: swap internal and external loops
system.time({
    res <- 
        mclapply(mc.cores=16, 1:resamp.iterations, function(n){
            res2 <- nmf(data.matrix[ ,index[n, ]], rank=2, seed = 12345)
            res3 <- nmf(data.matrix[ ,index[n, ]], rank=3, seed = 12345)
            res4 <- nmf(data.matrix[ ,index[n, ]], rank=4, seed = 12345)
            list(res2,res3,res4)
        })
})

## version 3: use previous calculation as starting point
##   ==> 30-40% reduction in computing time
system.time({
    res <- 
        mclapply(mc.cores=16, 1:resamp.iterations, function(n){
            res2 <- nmf(data.matrix[ ,index[n, ]], rank=2, seed = 12345)
            res3 <- nmf(data.matrix[ ,index[n, ]], rank=3, seed = res2)
            res4 <- nmf(data.matrix[ ,index[n, ]], rank=4, seed = res3)
            list(res2,res3,res4)
        })
})