使用gnuplot和awk计算应用程序加速

时间:2015-06-16 18:53:02

标签: performance awk gnuplot

问题在于:

加速公式:S(p)= T(1)/ T(p)=(一个过程的平均时间/ p 过程的平均时间)

有5个日志,其中一个想要提取信息。 cg.B.1.log包含一个进程的执行时间,因此我们计算获得T(1)的平均时间。其他日志文件包含2,4,8和16个进程的执行时间。还必须计算这些时间的平均值,因为它们是T(p)。

以下是计算平均值的代码:

system "awk 'NR==1{n=$0} {print n/$0}' tavg.dat > speedup.dat;"

计算加速的代码:

-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-
Start in 16:45:15--25/12/2014


 NAS Parallel Benchmarks 3.3 -- CG Benchmark

 Size:      75000
 Iterations:    75
 Number of active processes:     1
 Number of nonzeroes per row:       13
 Eigenvalue shift: .600E+02

   iteration           ||r||                 zeta
    1       0.30354859861452E-12    59.9994751578754
    2       0.11186435488267E-14    21.7627846142536
    3       0.11312258511928E-14    22.2876617043224
    4       0.11222160585284E-14    22.5230738188346
    5       0.11244234177219E-14    22.6275390653892
    6       0.11330434819384E-14    22.6740259189533
    7       0.11334259623050E-14    22.6949056826251
    8       0.11374839313647E-14    22.7044023166872
    9       0.11424877443039E-14    22.7087834345620
   10       0.11329475190566E-14    22.7108351397177
   11       0.11337364093482E-14    22.7118107121341
   12       0.11379928308864E-14    22.7122816240971
   13       0.11369453681794E-14    22.7125122663243
   14       0.11430390337015E-14    22.7126268007594
   15       0.11400318886400E-14    22.7126844161819
   16       0.11352091331197E-14    22.7127137461755
   17       0.11350923439124E-14    22.7127288402000
   18       0.11475378864565E-14    22.7127366848296
   19       0.11366777929028E-14    22.7127407981217
   20       0.11274243312504E-14    22.7127429721364
   21       0.11353930792856E-14    22.7127441294025
   22       0.11299685800278E-14    22.7127447493900
   23       0.11296405041170E-14    22.7127450834533
   24       0.11381975597887E-14    22.7127452643881
   25       0.11328127301663E-14    22.7127453628451
   26       0.11367332658939E-14    22.7127454166517
   27       0.11283372178605E-14    22.7127454461696
   28       0.11384734158863E-14    22.7127454624211
   29       0.11394011989719E-14    22.7127454713974
   30       0.11354294067640E-14    22.7127454763703
   31       0.11412988029103E-14    22.7127454791343
   32       0.11358088407717E-14    22.7127454806740
   33       0.11263266152515E-14    22.7127454815316
   34       0.11275183080286E-14    22.7127454820131
   35       0.11328306951409E-14    22.7127454822840
   36       0.11357880314891E-14    22.7127454824349
   37       0.11332687790488E-14    22.7127454825202
   38       0.11324108818137E-14    22.7127454825684
   39       0.11365065523777E-14    22.7127454825967
   40       0.11361185361321E-14    22.7127454826116
   41       0.11276519820716E-14    22.7127454826202
   42       0.11317183424878E-14    22.7127454826253
   43       0.11236007481770E-14    22.7127454826276
   44       0.11304065564684E-14    22.7127454826296
   45       0.11287791356431E-14    22.7127454826310
   46       0.11297028000133E-14    22.7127454826310
   47       0.11281236869666E-14    22.7127454826314
   48       0.11277254075548E-14    22.7127454826317
   49       0.11320327289847E-14    22.7127454826309
   50       0.11287655285563E-14    22.7127454826321
   51       0.11230503422400E-14    22.7127454826324
   52       0.11292089094944E-14    22.7127454826313
   53       0.11366728396408E-14    22.7127454826315
   54       0.11222618466968E-14    22.7127454826310
   55       0.11278193276516E-14    22.7127454826315
   56       0.11244624896030E-14    22.7127454826316
   57       0.11264508872685E-14    22.7127454826318
   58       0.11255583774760E-14    22.7127454826314
   59       0.11227129146723E-14    22.7127454826314
   60       0.11189480800173E-14    22.7127454826318
   61       0.11163241472678E-14    22.7127454826315
   62       0.11278839424218E-14    22.7127454826318
   63       0.11226804133008E-14    22.7127454826313
   64       0.11222456601361E-14    22.7127454826317
   65       0.11270879524310E-14    22.7127454826308
   66       0.11303771390006E-14    22.7127454826319
   67       0.11240101357287E-14    22.7127454826319
   68       0.11240278884391E-14    22.7127454826321
   69       0.11207748067718E-14    22.7127454826317
   70       0.11178755187571E-14    22.7127454826327
   71       0.11195935245649E-14    22.7127454826313
   72       0.11260715126337E-14    22.7127454826322
   73       0.11281677964997E-14    22.7127454826316
   74       0.11162340034815E-14    22.7127454826318
   75       0.11208709203921E-14    22.7127454826310
 Benchmark completed 
 VERIFICATION SUCCESSFUL 
 Zeta is     0.2271274548263E+02
 Error is    0.3128387698896E-15


 CG Benchmark Completed.
 Class           =                        B
 Size            =                    75000
 Iterations      =                       75
 Time in seconds =                    88.72
 Total processes =                        1
 Compiled procs  =                        1
 Mop/s total     =                   616.64
 Mop/s/process   =                   616.64
 Operation type  =           floating point
 Verification    =               SUCCESSFUL
 Version         =                      3.3
 Compile date    =              25 Dec 2014

 Compile options:
MPIF77       = mpif77
FLINK        = $(MPIF77)
FMPI_LIB     = -L/usr/lib/openmpi/lib -lmpi -lopen-rte -lo...
FMPI_INC     = -I/usr/lib/openmpi/include -I/usr/lib/openm...
FFLAGS       = -O
FLINKFLAGS   = -O
RAND         = randi8


 Please send the results of this run to:

 NPB Development Team 
 Internet: npb@nas.nasa.gov

 If email is not available, send this to:

 MS T27A-1
 NASA Ames Research Center
 Moffett Field, CA  94035-1000

 Fax: 650-604-3957


Finish in 16:46:46--25/12/2014
-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-
-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-
Start in 17:03:13--25/12/2014


 NAS Parallel Benchmarks 3.3 -- CG Benchmark

 Size:      75000
 Iterations:    75
 Number of active processes:     1
 Number of nonzeroes per row:       13
 Eigenvalue shift: .600E+02

   iteration           ||r||                 zeta
    1       0.30354859861452E-12    59.9994751578754
    2       0.11186435488267E-14    21.7627846142536
    3       0.11312258511928E-14    22.2876617043224
    4       0.11222160585284E-14    22.5230738188346
    5       0.11244234177219E-14    22.6275390653892
    6       0.11330434819384E-14    22.6740259189533
    7       0.11334259623050E-14    22.6949056826251
    8       0.11374839313647E-14    22.7044023166872
    9       0.11424877443039E-14    22.7087834345620
   10       0.11329475190566E-14    22.7108351397177
   11       0.11337364093482E-14    22.7118107121341
   12       0.11379928308864E-14    22.7122816240971
   13       0.11369453681794E-14    22.7125122663243
   14       0.11430390337015E-14    22.7126268007594
   15       0.11400318886400E-14    22.7126844161819
   16       0.11352091331197E-14    22.7127137461755
   17       0.11350923439124E-14    22.7127288402000
   18       0.11475378864565E-14    22.7127366848296
   19       0.11366777929028E-14    22.7127407981217
   20       0.11274243312504E-14    22.7127429721364
   21       0.11353930792856E-14    22.7127441294025
   22       0.11299685800278E-14    22.7127447493900
   23       0.11296405041170E-14    22.7127450834533
   24       0.11381975597887E-14    22.7127452643881
   25       0.11328127301663E-14    22.7127453628451
   26       0.11367332658939E-14    22.7127454166517
   27       0.11283372178605E-14    22.7127454461696
   28       0.11384734158863E-14    22.7127454624211
   29       0.11394011989719E-14    22.7127454713974
   30       0.11354294067640E-14    22.7127454763703
   31       0.11412988029103E-14    22.7127454791343
   32       0.11358088407717E-14    22.7127454806740
   33       0.11263266152515E-14    22.7127454815316
   34       0.11275183080286E-14    22.7127454820131
   35       0.11328306951409E-14    22.7127454822840
   36       0.11357880314891E-14    22.7127454824349
   37       0.11332687790488E-14    22.7127454825202
   38       0.11324108818137E-14    22.7127454825684
   39       0.11365065523777E-14    22.7127454825967
   40       0.11361185361321E-14    22.7127454826116
   41       0.11276519820716E-14    22.7127454826202
   42       0.11317183424878E-14    22.7127454826253
   43       0.11236007481770E-14    22.7127454826276
   44       0.11304065564684E-14    22.7127454826296
   45       0.11287791356431E-14    22.7127454826310
   46       0.11297028000133E-14    22.7127454826310
   47       0.11281236869666E-14    22.7127454826314
   48       0.11277254075548E-14    22.7127454826317
   49       0.11320327289847E-14    22.7127454826309
   50       0.11287655285563E-14    22.7127454826321
   51       0.11230503422400E-14    22.7127454826324
   52       0.11292089094944E-14    22.7127454826313
   53       0.11366728396408E-14    22.7127454826315
   54       0.11222618466968E-14    22.7127454826310
   55       0.11278193276516E-14    22.7127454826315
   56       0.11244624896030E-14    22.7127454826316
   57       0.11264508872685E-14    22.7127454826318
   58       0.11255583774760E-14    22.7127454826314
   59       0.11227129146723E-14    22.7127454826314
   60       0.11189480800173E-14    22.7127454826318
   61       0.11163241472678E-14    22.7127454826315
   62       0.11278839424218E-14    22.7127454826318
   63       0.11226804133008E-14    22.7127454826313
   64       0.11222456601361E-14    22.7127454826317
   65       0.11270879524310E-14    22.7127454826308
   66       0.11303771390006E-14    22.7127454826319
   67       0.11240101357287E-14    22.7127454826319
   68       0.11240278884391E-14    22.7127454826321
   69       0.11207748067718E-14    22.7127454826317
   70       0.11178755187571E-14    22.7127454826327
   71       0.11195935245649E-14    22.7127454826313
   72       0.11260715126337E-14    22.7127454826322
   73       0.11281677964997E-14    22.7127454826316
   74       0.11162340034815E-14    22.7127454826318
   75       0.11208709203921E-14    22.7127454826310
 Benchmark completed 
 VERIFICATION SUCCESSFUL 
 Zeta is     0.2271274548263E+02
 Error is    0.3128387698896E-15


 CG Benchmark Completed.
 Class           =                        B
 Size            =                    75000
 Iterations      =                       75
 Time in seconds =                    87.47
 Total processes =                        1
 Compiled procs  =                        1
 Mop/s total     =                   625.43
 Mop/s/process   =                   625.43
 Operation type  =           floating point
 Verification    =               SUCCESSFUL
 Version         =                      3.3
 Compile date    =              25 Dec 2014

 Compile options:
MPIF77       = mpif77
FLINK        = $(MPIF77)
FMPI_LIB     = -L/usr/lib/openmpi/lib -lmpi -lopen-rte -lo...
FMPI_INC     = -I/usr/lib/openmpi/include -I/usr/lib/openm...
FFLAGS       = -O
FLINKFLAGS   = -O
RAND         = randi8


 Please send the results of this run to:

 NPB Development Team 
 Internet: npb@nas.nasa.gov

 If email is not available, send this to:

     MS T27A-1
     NASA Ames Research Center
     Moffett Field, CA  94035-1000

     Fax: 650-604-3957


Finish in 17:04:43--25/12/2014
-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-/-

如何组合这两个命令,以便输出' speedup.dat'直接生成而不使用文件 tavg.dat

以下是文件的内容,所有日志文件的结构完全相同。为了缩写的目的,我只附加了前两个执行。

cg.B.1.log

88.3055
45.1482
37.7202
37.4035
53.777

tavg.dat

1
1.9559
2.34107
2.36089
1.64207

speedup.dat

<AppBarLayout>
    <CollapsingToolbarLayout>
        <ImageView/>
        <Toolbar/>
    </CollapsingToolbarLayout>
</AppBarLayout>

<android.support.v4.widget.NestedScrollView/> <!-- content here -->

2 个答案:

答案 0 :(得分:1)

您可以在一个处理所有日志文件的awk脚本中完成所有操作:

#!/usr/bin/awk -f

BEGIN { FS="=" }

lfname != FILENAME { lfname = FILENAME; split(FILENAME, a, "."); fnum=a[3] }

/Time in seconds/ { tsecs[fnum] += $2; tcnt[fnum]++ }
/Total processes/ { cp[fnum] = int($2) }

END {
    tavg1 = tsecs[1]/tcnt[1]
    for( k in tsecs ) {
        tavgk = tsecs[k]/tcnt[k]
        if( tavgk > 0 ) {
            print k OFS cp[k] OFS tavgk OFS tavg1/tavgk
        }
    }
}

如果你把它放在一个名为awk.script的文件中并使其chmod +x awk.script可执行,你可以在bash中运行它,如:

 ./awk.script cg.B.*.log

如果您正在使用GNU awk,则会对输出进行排序(可能需要额外的步骤以确保使用其他awk风格订购输出。)

在我生成第2和第3个文件的地方,输出如下:

1 1 88.095 1
2 2 68.095 1.29371
3 4 49.595 1.77629

其中未命名的列类似于:file number# processesavg per filespeedup。您可以通过将END块中的打印更改为print tavg1/tavgk来获得加速。

以下是剧本的细分:

  • BEGIN
  • 中使用更简单的字段分隔符
  • lfname != FILENAME - 将文件名中的文件编号解析为fnum,但仅在FILENAME更改时解析。
  • /Time in seconds/ - 使用tsecs键将值存储在tcntfnum数组中。使用int()函数从进程值中去除空格。
  • /Total processes/ - 使用cp键将流程存储在fnum数组中
  • END - 计算fnum 1的平均值tavg1,循环显示tsecs中的键,并按fnum键计算平均值{{ 1}}。当tavgk按上述方式打印输出时。

答案 1 :(得分:0)

你已经找到了所有困难的部分。您根本不需要tavg.dat文件。直接创建tavg(n)函数作为系统调用:

tavg(n) = system("awk 'BEGIN { FS = \"[ \\t]*=[ \\t]*\" }  \
/Time in seconds/ { s += $2; c++ } /Total processes/ { \
if (! CP) CP = $2 } END { print s/c }' cg.B.".n.".log")

speedup(n)函数

speedup(n)=tavg(n)/tavg(1)

现在您可以set print写入文件:

set print "speedup.dat"
do for [i=1:5] {
print speedup(i)
}
unset print