Question

我必须通过重复测量个体的几个变量来检查大型数据库。由于我可以有超过300万的观察，我想至少删除我确定数据输入错误的数据。

连续变量

例如，关注可变重量（例如下面的数据框），我知道在一次观察和下一次观察之间，个体不能将体重减轻40％以上。我怎样才能检测出体重减轻较多的观察结果，如第3次观察个体“2”，其体重从30克减少到3克。

分类变量

例如，关于个人的状态。一个人可以被分类为3种状态（例如“少年”，“成年非种鸡”或“成年种鸡”;分别为1,2和3）。我知道一个人如果是成年人（“2”或“3”）就不能成为少年（“1”），但有可能在3 - > 2之间过渡。在这个特殊情况下，我想检测观察9，其中个体“3”被归类为“少年”，但在之前的观察中被归类为“成人”。

MyProgress {
    id: progress
    onProgressChanged:{
        progressAnimation.to = progress
        progressAnimation.duration =
                (progress - myProgressBar.value) * 100 // change 100 to speed up or slow down animation.
        progressAnimation.restart()
    }
}
NumberAnimation {
    id: progressAnimation
    target: myProgressBar
    property: "value"
}

你知道我怎么能解决这两种错误？

Answer 1

根据您的说明并仅根据您上面提到的“问题”尝试：

Individuals <- c(1,1,1,2,2,2,3,3,3)
Weight <- c(10, 14, 20, 15, 30, 3, 12, 34, 30)
Week <- rep(1:3, 3)
Status <- c(1, 2, 3, 2, 3, 3, 2, 3, 1)
df <- as.data.frame (cbind(Individuals, Weight, Week, Status))

library(dplyr)

df %>%
  group_by(Individuals) %>%      ## for each individual
  mutate(WeightReduce = 1-Weight/dplyr::lag(Weight, default = Weight[1])) %>%  ## calculate the weight reduce (negative numbers here mean weight increase)
  ungroup() %>%                  ## forget the grouping
  mutate(flag = ifelse(WeightReduce >= 0.4 | dplyr::lag(Status, default = Status[1]) %in% 2:3 & Status == 1, 1, 0))  ## flag errors based on filters


#    Individuals Weight  Week Status WeightReduce  flag
#          (dbl)  (dbl) (dbl)  (dbl)        (dbl) (dbl)
# 1           1     10     1      1    0.0000000     0
# 2           1     14     2      2   -0.4000000     0
# 3           1     20     3      3   -0.4285714     0
# 4           2     15     1      2    0.0000000     0
# 5           2     30     2      3   -1.0000000     0
# 6           2      3     3      3    0.9000000     1
# 7           3     12     1      2    0.0000000     0
# 8           3     34     2      3   -1.8333333     0
# 9           3     30     3      1    0.1176471     1

Answer 2

您可以使用data.table包来计算体重变化率和青少年异常，然后根据这两个标准进行过滤：

library(data.table)

setDT(df)[,c('continuous', 'categorical'):=list(
              c(0,diff(Weight)/head(Weight, -1)),  # rate of weight change per individual
              Status==1 & c(F,diff(Status)<0)),Individuals][ 
          continuous>=-0.4 & !categorical,][]

#   Individuals Weight Week Status    change continuous categorical
#1:           1     10    1      1 0.0000000  0.0000000       FALSE
#2:           1     14    2      2 0.4000000  0.4000000       FALSE
#3:           1     20    3      3 0.4285714  0.4285714       FALSE
#4:           2     15    1      2 0.0000000  0.0000000       FALSE
#5:           2     30    2      3 1.0000000  1.0000000       FALSE
#6:           3     12    1      2 0.0000000  0.0000000       FALSE
#7:           3     34    2      3 1.8333333  1.8333333       FALSE

Answer 3

我希望这会有所帮助。

library(data.table)
  library(zoo)
  df <- data.table(df)
  # used to check percentage change in weight variable
  calcreduction <- function(x){
    res <- diff(x)/x[-length(x)]
    return(c(0,res))
  }
  # this will make it easy to get rid of values where WeightReduction < -.4

  #function used to assign combination type
  # you can have 11,12,13,22,23,32,33 or 21,31. The latter are "bad"
  getcomb <- function(x){
    res <- rbind(c(0,0),rollapply(x,2,paste))
    return(paste(res[,1],res[,2],sep=""))
  } 
  # this will make it easy to get rid of values where the Status change is no good

  # you can just pull the new vectors and then use logic
  # to decide what you want to do with these values
  res <- df[,list("WeightReduction"=calcreduction(Weight),
                  "StatusChange"=getcomb(Status),Weight,Week,Status),by=Individuals]

> res
   Individuals WeightReduction StatusChange Weight Week Status
1:           1       0.0000000           00     10    1      1
2:           1       0.4000000           12     14    2      2
3:           1       0.4285714           23     20    3      3
4:           2       0.0000000           00     15    1      2
5:           2       1.0000000           23     30    2      3
6:           2      -0.9000000           33      3    3      3
7:           3       0.0000000           00     12    1      2
8:           3       1.8333333           23     34    2      3
9:           3      -0.1176471           31     30    3      1

从数据帧中的重复测量中检测不可能的数据输入错误

3 个答案: