我希望(1)按一个变量(State
)分组数据,(2)在每个组中找到另一个变量(Employees
)的最小值行,以及(3)提取整行。
(1)和(2)很容易单行,我觉得(3)应该也是,但我无法得到它。
以下是一个示例数据集:
> data
State Company Employees
1 AK A 82
2 AK B 104
3 AK C 37
4 AK D 24
5 RI E 19
6 RI F 118
7 RI G 88
8 RI H 42
data <- structure(list(State = structure(c(1L, 1L, 1L, 1L, 2L, 2L, 2L,
2L), .Label = c("AK", "RI"), class = "factor"), Company = structure(1:8, .Label = c("A",
"B", "C", "D", "E", "F", "G", "H"), class = "factor"), Employees = c(82L,
104L, 37L, 24L, 19L, 118L, 88L, 42L)), .Names = c("State", "Company",
"Employees"), class = "data.frame", row.names = c(NA, -8L))
使用min
aggregate
> aggregate(Employees ~ State, data, function(x) min(x))
State Employees
1 AK 24
2 RI 19
...或data.table
:
> library(data.table)
> DT <- data.table(data)
> DT[ , list(Employees = min(Employees)), by = State]
State Employees
1: AK 24
2: RI 19
但是如何提取与这些min
值对应的整行,即在结果中还包括Company
?
答案 0 :(得分:46)
稍微优雅一点:
library(data.table)
DT[ , .SD[which.min(Employees)], by = State]
State Company Employees
1: AK D 24
2: RI E 19
比使用.SD
更不优雅,但速度更快(对于包含多个组的数据):
DT[DT[ , .I[which.min(Employees)], by = State]$V1]
此外,只需将表达式which.min(Employees)
替换为Employees == min(Employees)
,如果您的数据集具有多个相同的最小值,并且您希望对所有这些值进行分组。
答案 1 :(得分:44)
dplyr
解决方案:
library(dplyr)
data %>%
group_by(State) %>%
slice(which.min(Employees))
答案 2 :(得分:23)
由于这是Google的热门话题,我想我会添加一些我认为有用的其他选项。这个想法基本上是通过Employees
安排一次,然后根据State
使用data.table
library(data.table)
unique(setDT(data)[order(Employees)], by = "State")
# State Company Employees
# 1: RI E 19
# 2: AK D 24
或者,我们也可以先订购,然后再分组.SD
。这两个操作都在重新发布的数据中进行了优化。order
似乎触发data.table:::forderv
,而.SD[1L]
触发Gforce
setDT(data)[order(Employees), .SD[1L], by = State, verbose = TRUE] # <- Added verbose
# order optimisation is on, i changed from 'order(...)' to 'forder(DT, ...)'.
# i clause present and columns used in by detected, only these subset: State
# Finding groups using forderv ... 0 sec
# Finding group sizes from the positions (can be avoided to save RAM) ... 0 sec
# Getting back original order ... 0 sec
# lapply optimization changed j from '.SD[1L]' to 'list(Company[1L], Employees[1L])'
# GForce optimized j to 'list(`g[`(Company, 1L), `g[`(Employees, 1L))'
# Making each group and running j (GForce TRUE) ... 0 secs
# State Company Employees
# 1: RI E 19
# 2: AK D 24
或dplyr
library(dplyr)
data %>%
arrange(Employees) %>%
distinct(State, .keep_all = TRUE)
# State Company Employees
# 1 RI E 19
# 2 AK D 24
借用@Khashaas非常棒的答案(为了处理多个匹配而对mult = "first"
进行小修改)的另一个有趣的想法是首先找到每个组的最小值,然后再执行二进制连接。这样做的好处是可以利用data.tables gmin
函数(跳过评估开销)和二进制连接功能
tmp <- setDT(data)[, .(Employees = min(Employees)), by = State]
data[tmp, on = .(State, Employees), mult = "first"]
# State Company Employees
# 1: AK D 24
# 2: RI E 19
一些基准
library(data.table)
library(dplyr)
library(plyr)
library(stringi)
library(microbenchmark)
set.seed(123)
N <- 1e6
data <- data.frame(State = stri_rand_strings(N, 2, '[A-Z]'),
Employees = sample(N*10, N, replace = TRUE))
DT <- copy(data)
setDT(DT)
DT2 <- copy(DT)
str(DT)
str(DT2)
microbenchmark("(data.table) .SD[which.min]: " = DT[ , .SD[which.min(Employees)], by = State],
"(data.table) .I[which.min]: " = DT[DT[ , .I[which.min(Employees)], by = State]$V1],
"(data.table) order/unique: " = unique(DT[order(Employees)], by = "State"),
"(data.table) order/.SD[1L]: " = DT[order(Employees), .SD[1L], by = State],
"(data.table) self join (on):" = {
tmp <- DT[, .(Employees = min(Employees)), by = State]
DT[tmp, on = .(State, Employees), mult = "first"]},
"(data.table) self join (setkey):" = {
tmp <- DT2[, .(Employees = min(Employees)), by = State]
setkey(tmp, State, Employees)
setkey(DT2, State, Employees)
DT2[tmp, mult = "first"]},
"(dplyr) slice(which.min): " = data %>% group_by(State) %>% slice(which.min(Employees)),
"(dplyr) arrange/distinct: " = data %>% arrange(Employees) %>% distinct(State, .keep_all = TRUE),
"(dplyr) arrange/group_by/slice: " = data %>% arrange(Employees) %>% group_by(State) %>% slice(1),
"(plyr) ddply/which.min: " = ddply(data, .(State), function(x) x[which.min(x$Employees),]),
"(base) by: " = do.call(rbind, by(data, data$State, function(x) x[which.min(x$Employees), ])))
# Unit: milliseconds
# expr min lq mean median uq max neval cld
# (data.table) .SD[which.min]: 119.66086 125.49202 145.57369 129.61172 152.02872 267.5713 100 d
# (data.table) .I[which.min]: 12.84948 13.66673 19.51432 13.97584 15.17900 109.5438 100 a
# (data.table) order/unique: 52.91915 54.63989 64.39212 59.15254 61.71133 177.1248 100 b
# (data.table) order/.SD[1L]: 51.41872 53.22794 58.17123 55.00228 59.00966 145.0341 100 b
# (data.table) self join (on): 44.37256 45.67364 50.32378 46.24578 50.69411 137.4724 100 b
# (data.table) self join (setkey): 14.30543 15.28924 18.63739 15.58667 16.01017 106.0069 100 a
# (dplyr) slice(which.min): 82.60453 83.64146 94.06307 84.82078 90.09772 186.0848 100 c
# (dplyr) arrange/distinct: 344.81603 360.09167 385.52661 379.55676 395.29463 491.3893 100 e
# (dplyr) arrange/group_by/slice: 367.95924 383.52719 414.99081 397.93646 425.92478 557.9553 100 f
# (plyr) ddply/which.min: 506.55354 530.22569 568.99493 552.65068 601.04582 727.9248 100 g
# (base) by: 1220.38286 1291.70601 1340.56985 1344.86291 1382.38067 1512.5377 100 h
答案 3 :(得分:7)
基函数by
通常用于处理data.frames中的块数据。例如
by(data, data$State, function(x) x[which.min(x$Employees), ] )
它会返回列表中的数据,但您可以使用
将其折叠do.call(rbind, by(data, data$State, function(x) x[which.min(x$Employees), ] ))
答案 4 :(得分:3)
更正了plyr
解决方案:
ddply(df, .(State), function(x) x[which.min(x$Employees),])
# State Company Employees
# 1 AK D 24
# 2 RI E 19
答案 5 :(得分:1)
在 base 中,您可以使用ave
获取每个组的min
并将其与Employees
进行比较,并获得一个逻辑向量来对{{1} }。
data.frame
或者已经在函数中进行了比较。
data[data$Employees == ave(data$Employees, data$State, FUN=min),]
# State Company Employees
#4 AK D 24
#5 RI E 19
答案 6 :(得分:0)
使用 from pyspark.sql import SparkSession
def patch_pyspark_accumulators():
from inspect import getsource
import pyspark.accumulators as pa
exec(getsource(pa._start_update_server).replace("localhost", "127.0.0.1"), pa.__dict__)
patch_pyspark_accumulators()
spark = SparkSession.builder.getOrCreate()
<platform name="ios">
<preference name="scheme" value="app" />
<preference name="hostname" value="localhost" />
</platform>