R stats :: sd()与arma :: stddev()与Rcpp实现的性能

时间:2014-06-16 22:33:32

标签: c++ r performance rcpp armadillo

为了完成我的C ++ / Rcpp编程,我尝试了实现(样本)标准偏差函数:

#include <Rcpp.h>
#include <vector>
#include <cmath>
#include <numeric>

// [[Rcpp::export]]
double cppSD(Rcpp::NumericVector rinVec)
{
  std::vector<double> inVec(rinVec.begin(),rinVec.end());
  int n = inVec.size();
  double sum = std::accumulate(inVec.begin(), inVec.end(), 0.0);
  double mean = sum / inVec.size();

  for(std::vector<double>::iterator iter = inVec.begin();
      iter != inVec.end(); ++iter){
        double temp;
        temp= (*iter - mean)*(*iter - mean);
        *iter = temp;
      }

  double sd = std::accumulate(inVec.begin(), inVec.end(), 0.0);
  return std::sqrt( sd / (n-1) );
}

我还决定测试Armadillo库中的stddev函数,考虑到它可以在向量上调用:

#include <RcppArmadillo.h>

// [[Rcpp::depends(RcppArmadillo)]]

using namespace Rcpp;

// [[Rcpp::export]]

double armaSD(arma::colvec inVec)
{
  return arma::stddev(inVec);
}

然后我针对一些不同大小的向量,对基本R函数sd()对这两个函数进行了基准测试:

Rcpp::sourceCpp('G:/CPP/armaSD.cpp')
Rcpp::sourceCpp('G:/CPP/cppSD.cpp')
require(microbenchmark)
##
## sample size = 1,000: armaSD() < cppSD() < sd()
X <- rexp(1000)
microbenchmark(armaSD(X),sd(X), cppSD(X)) 
#Unit: microseconds
#      expr    min     lq median     uq    max neval
# armaSD(X)  4.181  4.562  4.942  5.322 12.924   100
#     sd(X) 17.865 19.766 20.526 21.287 86.285   100
#  cppSD(X)  4.561  4.941  5.321  5.701 29.269   100
##
## sample size = 10,000: armaSD() < cppSD() < sd()
X <- rexp(10000)
microbenchmark(armaSD(X),sd(X), cppSD(X))
#Unit: microseconds
#      expr    min     lq  median      uq     max neval
# armaSD(X) 24.707 25.847 26.4175 29.6490  52.455   100
#     sd(X) 51.315 54.356 55.8760 61.1980 100.730   100
#  cppSD(X) 26.608 28.128 28.8885 31.7395 114.413   100
##
## sample size = 25,000: armaSD() < cppSD() < sd()
X <- rexp(25000)
microbenchmark(armaSD(X),sd(X), cppSD(X))
#Unit: microseconds
#      expr     min       lq  median      uq     max neval
# armaSD(X)  66.900  67.6600  68.040  76.403 155.845   100
#     sd(X) 108.332 111.5625 122.016 125.817 169.910   100
#  cppSD(X)  70.320  71.0805  74.692  80.203 102.250   100
##
## sample size = 50,000: cppSD() < sd() < armaSD()
X <- rexp(50000)
microbenchmark(armaSD(X),sd(X), cppSD(X))
#Unit: microseconds
#      expr     min       lq   median      uq     max neval
# armaSD(X) 249.733 267.4085 297.8175 337.729 642.388   100
#     sd(X) 203.740 229.3975 240.2300 260.186 303.709   100
#  cppSD(X) 162.308 185.1140 239.6600 256.575 290.405   100
##
## sample size = 75,000: sd() < cppSD() < armaSD()
X <- rexp(75000)
microbenchmark(armaSD(X),sd(X), cppSD(X))
#Unit: microseconds
#      expr     min       lq   median       uq     max neval
# armaSD(X) 445.110 479.8900 502.5070 520.5625 642.388   100
#     sd(X) 310.931 334.8780 354.0735 379.7310 429.146   100
#  cppSD(X) 346.661 380.8715 400.6370 424.0140 501.747   100

对于较小的样本,我的C ++函数cppSD()stats::sd()快,但是对于较大尺寸的向量较慢,因为stats::sd()被矢量化,因此我感到非常惊讶。但是,我没想到arma::stddev()函数会出现性能下降,因为它似乎也是以矢量化方式运行的。我使用arma::stdev()的方式是否存在问题,或者只是stats::sd()以这种方式编写(C我假设)它可以更有效地处理更大的向量?任何意见都将不胜感激。


更新

虽然我的问题最初是关于arma::stddev的正确使用,而不是尝试找到最有效的方法来计算样本标准偏差,但是看到{{1}这是非常有趣的。糖功能表现得很好。为了让事情变得更有趣,我将下面的Rcpp::sdarma::stddev函数与我从JJ Allaire的两个Rcpp图库帖子改编的Rcpp::sd版本进行了基准测试 - {{ 3}}和here

RcppParallel

这是在我的笔记本电脑上运行64位Linux,配备i5-4200U CPU @ 1.60GHz处理器;但我猜测library(microbenchmark) set.seed(123) x <- rnorm(5.5e06) ## Res <- microbenchmark( armaSD(x), par_sd(x), sd_sugar(x), times=500L, control=list(warmup=25)) ## R> print(Res) Unit: milliseconds expr min lq mean median uq max neval armaSD(x) 24.486943 24.960966 26.994684 25.255584 25.874139 123.55804 500 par_sd(x) 8.130751 8.322682 9.136323 8.429887 8.624072 22.77712 500 sd_sugar(x) 13.713366 13.984638 14.628911 14.156142 14.401138 32.81684 500 par_sd之间的差异在Windows机器上会不那么重要。

sugar_sd版本的代码(相当长,并且需要与RcppParallel仿函数的重载operator()中使用的lambda表达式的C ++ 11兼容编译器:

InnerProduct

3 个答案:

答案 0 :(得分:15)

你在如何实例化Armadillo对象时犯了一个微妙的错误 - 这会导致副本,从而降低性能。

使用const arma::colvec & invec的界面代替,一切都很好:

R> sourceCpp("/tmp/sd.cpp")

R> library(microbenchmark)

R> X <- rexp(500)

R> microbenchmark(armaSD(X), armaSD2(X), sd(X), cppSD(X))
Unit: microseconds
       expr    min      lq  median      uq    max neval
  armaSD(X)  3.745  4.0280  4.2055  4.5510 19.375   100
 armaSD2(X)  3.305  3.4925  3.6400  3.9525  5.154   100
      sd(X) 22.463 23.6985 25.1525 26.0055 52.457   100
   cppSD(X)  3.640  3.9495  4.2030  4.8620 13.609   100

R> X <- rexp(5000)

R> microbenchmark(armaSD(X), armaSD2(X), sd(X), cppSD(X))
Unit: microseconds
       expr    min      lq  median      uq    max neval
  armaSD(X) 18.627 18.9120 19.3245 20.2150 34.684   100
 armaSD2(X) 14.583 14.9020 15.1675 15.5775 22.527   100
      sd(X) 54.507 58.8315 59.8615 60.4250 84.857   100
   cppSD(X) 18.585 19.0290 19.3970 20.5160 22.174   100

R> X <- rexp(50000)

R> microbenchmark(armaSD(X), armaSD2(X), sd(X), cppSD(X))
Unit: microseconds
       expr     min      lq  median      uq     max neval
  armaSD(X) 186.307 187.180 188.575 191.825 405.775   100
 armaSD2(X) 142.447 142.793 143.207 144.233 155.770   100
      sd(X) 382.857 384.704 385.223 386.075 405.713   100
   cppSD(X) 181.601 181.895 182.279 183.350 194.588   100
R> 

基于我的代码版本,其中所有内容都是一个文件,armaSD2按照我的建议定义 - 从而获得了成功。

#include <RcppArmadillo.h>

// [[Rcpp::depends(RcppArmadillo)]]  
#include <vector>
#include <cmath>
#include <numeric>

// [[Rcpp::export]]
double cppSD(Rcpp::NumericVector rinVec) {
  std::vector<double> inVec(rinVec.begin(),rinVec.end());
  int n = inVec.size();
  double sum = std::accumulate(inVec.begin(), inVec.end(), 0.0);
  double mean = sum / inVec.size();

  for(std::vector<double>::iterator iter = inVec.begin();
      iter != inVec.end(); 
      ++iter){
    double temp = (*iter - mean)*(*iter - mean);
    *iter = temp;
  }

  double sd = std::accumulate(inVec.begin(), inVec.end(), 0.0);
  return std::sqrt( sd / (n-1) );
}

// [[Rcpp::export]]      
double armaSD(arma::colvec inVec) {
  return arma::stddev(inVec);
}

//  [[Rcpp::export]]    
double armaSD2(const arma::colvec & inVec) { return arma::stddev(inVec); }

/*** R
library(microbenchmark)
X <- rexp(500)
microbenchmark(armaSD(X), armaSD2(X), sd(X), cppSD(X)) 

X <- rexp(5000)
microbenchmark(armaSD(X), armaSD2(X), sd(X), cppSD(X)) 

X <- rexp(50000)    
microbenchmark(armaSD(X), armaSD2(X), sd(X), cppSD(X))
*/

答案 1 :(得分:2)

我认为用Rcpp糖建立的sd函数效率更高。请参阅以下代码:

   #include <RcppArmadillo.h>
   //[[Rcpp::depends(RcppArmadillo)]]
   #include <vector>
   #include <cmath>
   #include <numeric>
   using namespace Rcpp;
   //[[Rcpp::export]]                                                                                                                                                         
  double sd_cpp(NumericVector& xin){
 std::vector<double> xres(xin.begin(),xin.end());
 int n=xres.size();
 double sum=std::accumulate(xres.begin(),xres.end(),0.0);
 double mean=sum/n;
 for(std::vector<double>::iterator iter=xres.begin();iter!=xres.end();++iter){
   double tmp=(*iter-mean)*(*iter-mean);
   *iter=tmp;
 }
   double sd=std::accumulate(xres.begin(),xres.end(),0.0);
   return std::sqrt(sd/(n-1));
 }
  //[[Rcpp::export]]
 double sd_arma(arma::colvec& xin){
 return arma::stddev(xin);
}
 //[[Rcpp::export]]
 double sd_sugar(NumericVector& xin){
 return sd(xin);
}

> sourcecpp("sd.cpp")

> microbenchmark(sd(X),sd_cpp(X),sd_arma(X),sd_sugar(X))
   Unit: microseconds
      expr    min      lq     mean  median      uq     max neval
      sd(X) 47.655 49.4120 51.88204 50.5395 51.1950 113.643   100
  sd_cpp(X) 28.145 28.4410 29.01541 28.6695 29.4570  37.118   100
  sd_arma(X) 23.706 23.9615 24.65931 24.1955 24.9520  50.375   100
 sd_sugar(X) 19.197 19.478 20.38872 20.0785 21.2015  28.664   100

答案 2 :(得分:0)

关于Rcpp函数的2个问题:

1)您希望没有均值的标准偏差的可能性有多大?如果要求没有平均值的SD的情况很少见,为什么不返回两者,而不是让R用户进行2次函数调用,实际上两次计算平均值就可以了。

2)有什么理由在函数内部克隆载体?

using namespace Rcpp;
// [[Rcpp::plugins(cpp14)]]

// [[Rcpp::export]]
NumericVector cppSD(NumericVector rinVec)
{
  double n = (double)rinVec.size();
  double sum = 0;
  for (double& v : rinVec)
     sum += v;
  double mean = sum / n;
  double varianceNumerator = 0;
  for(double& v : rinVec)
     varianceNumerator += (v - mean) * (v - mean);
  return NumericVector::create(_["std.dev"] = sqrt(varianceNumerator/ (n - 1.0)),
                               _["mean"] = mean);
}