Question

我有一个大数据集。我想分成“n”个子数据集，每个子数据集具有相同的大小“s”。但是，如果不能按数字整除，则最后一个数据集可能小于其他数据集。并将它们作为csv文件输出到工作目录。

让我们说以下小例子：

set.seed(1234)
mydf <- data.frame (matrix(sample(1:10, 130, replace = TRUE), ncol = 13))
mydf

   X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13
1   3  7  1  9  6  4  7  5  8   2   2   2   8
2   5  3  4  6  9  5  3 10  5   8  10   2  10
3   4  6 10  4  4  6  3  4  2   9   9   2   9
4  10 10  9  4  3  7  7  7 10   6   7  10   2
5  10  3  9  3  2 10  9  6  4   4   4   6   3
6   7  2  8  7  5  5 10 10  9   3   7   8   4
7   3  2  2  7 10  9  2  2 10   1   1  10   4
8   3  9  9  7  3  1  7  6 10   3  10   3   2
9   9  3  6  9  3  2  2  3  4   2   9  10  10
10  6  4  3  3  5  9  3  9 10   7   4   6  10

我想创建一个函数，将数据集随机分成n个子集（在这种情况下说大小为3，因为有13列 - 最后一个数据集将有1列剩余4个，每个有3个）并输出为文本将文件作为单独的数据集。

这是我做的：

set.seed(123)
reshuffled <- sample(1:length(mydf),length(mydf), replace = FALSE)
# just crazy manual divide 
group1 <- reshuffled[1:3]; group2 <- reshuffled[4:6]; group3 <- reshuffled[7:9]
group4 <- reshuffled[10:12]; group5 <-  reshuffled[13]

# just manual 
data1 <- mydf[,group1]; data2 <- mydf[,group2]; ....so on;
# I want to write dimension of dataset at fist row of each dataset 
cat (dim(data1))
write.csv(data1, "data1.csv");  write.csv(data2, "data2.csv"); .....so on

是否可以循环过程，因为我必须生成100个子数据集？

Answer 1

也许有更简洁的解决方案，但您可以尝试以下方法：

mydf <- data.frame (matrix(sample(1:10, 130, replace = TRUE), ncol = 13))

## Number of columns for each sub-dataset
size <- 3

nb.cols <- ncol(mydf)
nb.groups <- nb.cols %/% size
reshuffled <- sample.int(nb.cols, replace=FALSE)
groups <- c(rep(1:nb.groups, each=size), rep(nb.groups+1, nb.cols %% size))
dfs <- lapply(split(reshuffled, groups), function(v) mydf[,v,drop=FALSE])

for (i in 1:length(dfs)) write.csv(dfs[[i]], file=paste("data",i,".csv",sep=""))

Answer 2

只是为了好玩，可能比朱巴的慢

mydf <- data.frame (matrix(sample(1:10, 130, replace = TRUE), ncol = 13))
size <- 3
by(t(mydf), 
   INDICES=sample(as.numeric(gl((ncol(mydf) %/% size) + 1, size, ncol(mydf))), 
                  ncol(mydf), 
                  replace=FALSE), 
   FUN=function(x) write.csv(t(x), paste(rownames(x), collapse='-'), row.names=F))

Answer 3

为了将'mydf'分成几乎相等的部分，我从中获取灵感这个问题和相应的答案： link

它创建的分区大小最小和之间的差异最大的分区尽可能小。在此示例中，此差异等于1.示例：

分区方法1 - 使用'floor'功能（此处显示无可重现的代码）。通过随后的样本层（100/7）=前6次迭代的14个索引，在7个几乎相等的部分/加数中划分100行。第7个元素是余数。这会产生：

14,14,14,14,14,14,16。总和= 100，最大差异= 2

分区方法2 - 使用'ceiling'功能（此处显示无可重现的代码）。使用'ceiling'函数而不是'floor'函数可以得到类似的结果：

15,15,15,15,15,15,10。总和= 100，最大差异= 5

分区方法3 - 使用上面参考的公式。使用以下过程时，分区大小的向量（'sequence_diff'）为：

14,14,14,15,14,14,15。Sum = 100，max difference = 1

R-代码：

set.seed(1234)
#I increased the number of rows in the data frame to 100
mydf <- data.frame (matrix(sample(x = 1:100, size = 1300, replace = TRUE), 
                    ncol = 13))

index_list      <- list()       #Will store the indices for all partitions
indices         <- 1:nrow(mydf) #Initially contains all indices for the dataset 'mydf'
numb_partitions <- 7            #Specifies the number of partitions

sequence <- floor(((nrow(mydf)*1:numb_partitions)/numb_partitions))
sequence <- c(0, sequence)

#'sequence_diff' will contain the number of instances for each partition.
sequence_diff <- vector()
for(j in 1:numb_partitions){
    sequence_diff[j] <- sequence[j+1] - sequence[j]   
}  

#Inspect 'sequence_diff' and verify it's elements sum up to the total 
#number of rows in 'mydf' (100).
> sequence_diff
[1] 14 14 14 15 14 14 15
> sum(sequence_diff)
[1] 100 #Correct!

for(i in 1:numb_partitions){

  #Use a different seed for each sampling iteration.
  set.seed(seed = i)

  #Sample from object 'indices' of size 1/'numb_partitions'
  indices_partition <- sample(x = indices, 
                              size = sequence_diff[i], 
                              replace = FALSE)

  #Remove the selected indices from 'indices' so these indices will not be 
  #selected in successive iterations.
  indices           <- setdiff(x = indices, y = indices_partition)

  #Store the indices for the i-th iteration in the list 'index_list'. This 
  #is just to verify later that 
  #the procedure has divided all indices in 'numb_partitions' disjunct sets.
  index_list[[i]]   <- indices_partition

  #Dynamically create a new object that is named 'mydfx' in which x is the 
  #i-th partition. 
  assign(x = paste0("mydf", i), value = mydf[indices_partition,])

  write.csv(x = get(x = paste0("mydf", i)),  #Dynamically get the object from environment.
            file = paste0("mydf", i,".csv"), #Dynamically assgin a name to the csv-file.
            sep = ",", 
            col.names = T, 
            row.names = FALSE    
}

#Check whether all index subsets are mutually exclusive: union should have 100 
#unique elements. 
length(unique(unlist(index_list)))
[1] 100 #Correct!

将数据集拆分为多个数据集，其中r中包含随机列

3 个答案: