Question

我今天测试了TensorFlow通信时间，但是在具有不同批量大小的模型中通信时间不相等。谁能告诉我为什么？

我在/tensorflow/tensorflow/core/common_runtime/executor.cc中添加了登录测试文件是/tensorflow/tensorflow/tools/dist_test/python/mnist_replica.py

＆＃34; sm_w＆＃34;的大小对于任何批量大小都是不变的。日志看起来像这样

批量大小= 30000

＆＃13;

Sync Start sm_w/read_S78 _Send NC_SEND 1477676255.628095
Sync End sm_w/read_S78 _Send NC_SEND 1477676255.628133
time= 0.000038
Async Start sm_w/read_S79 _Recv NC_RECV 1477676255.871527
Async End sm_w/read_S79 _Recv NC_RECV 1477676255.900471
time= 0.028944
Sync Start sm_w/read_S78 _Send NC_SEND 1477676256.554135
Sync End sm_w/read_S78 _Send NC_SEND 1477676256.554159
time= 0.000024
Async Start sm_w/read_S79 _Recv NC_RECV 1477676256.811894
Async End sm_w/read_S79 _Recv NC_RECV 1477676256.835336
time= 0.023442
Sync Start sm_w/read_S78 _Send NC_SEND 1477676257.279976
Sync End sm_w/read_S78 _Send NC_SEND 1477676257.280021
time= 0.000045
Async Start sm_w/read_S79 _Recv NC_RECV 1477676257.517868
Async End sm_w/read_S79 _Recv NC_RECV 1477676257.532140
time= 0.014272
Sync Start sm_w/read_S78 _Send NC_SEND 1477676257.949458
Sync End sm_w/read_S78 _Send NC_SEND 1477676257.949482
time= 0.000024
Async Start sm_w/read_S79 _Recv NC_RECV 1477676258.222357
Async End sm_w/read_S79 _Recv NC_RECV 1477676258.236502
time= 0.014145

＆＃13;

批量大小= 10000

＆＃13;

Sync Start sm_w/read_S78 _Send NC_SEND 1477669486.223020
Sync End sm_w/read_S78 _Send NC_SEND 1477669486.223471
time= 0.000451
Async Start sm_w/read_S79 _Recv NC_RECV 1477669486.322305
Async End sm_w/read_S79 _Recv NC_RECV 1477669486.327682
time= 0.005377
Sync Start sm_w/read_S78 _Send NC_SEND 1477669486.564465
Sync End sm_w/read_S78 _Send NC_SEND 1477669486.564487
time= 0.000022
Async Start sm_w/read_S79 _Recv NC_RECV 1477669486.662599
Async End sm_w/read_S79 _Recv NC_RECV 1477669486.668082
time= 0.005483
Sync Start sm_w/read_S78 _Send NC_SEND 1477669486.745281
Sync End sm_w/read_S78 _Send NC_SEND 1477669486.745304
time= 0.000023
Async Start sm_w/read_S79 _Recv NC_RECV 1477669486.837152
Async End sm_w/read_S79 _Recv NC_RECV 1477669486.842311
time= 0.005159
Sync Start sm_w/read_S78 _Send NC_SEND 1477669486.927430
Sync End sm_w/read_S78 _Send NC_SEND 1477669486.927454
time= 0.000024
Async Start sm_w/read_S79 _Recv NC_RECV 1477669487.029626
Async End sm_w/read_S79 _Recv NC_RECV 1477669487.035640
time= 0.006014

＆＃13;

批量大小= 1000

＆＃13;

Sync Start sm_w/read_S78 _Send NC_SEND 1477669435.974999
Sync End sm_w/read_S78 _Send NC_SEND 1477669435.975094
time= 0.000095
Async Start sm_w/read_S79 _Recv NC_RECV 1477669436.026246
Async End sm_w/read_S79 _Recv NC_RECV 1477669436.029056
time= 0.002810
Sync Start sm_w/read_S78 _Send NC_SEND 1477669436.182368
Sync End sm_w/read_S78 _Send NC_SEND 1477669436.182408
time= 0.000040
Sync Start sm_w/read_S78 _Send NC_SEND 1477669436.213410
Sync End sm_w/read_S78 _Send NC_SEND 1477669436.213443
time= 0.000033
Async Start sm_w/read_S79 _Recv NC_RECV 1477669436.233330
Async End sm_w/read_S79 _Recv NC_RECV 1477669436.235818
time= 0.002488
Sync Start sm_w/read_S78 _Send NC_SEND 1477669436.263110
Sync End sm_w/read_S78 _Send NC_SEND 1477669436.263131
time= 0.000021
Async Start sm_w/read_S79 _Recv NC_RECV 1477669436.267269
Async End sm_w/read_S79 _Recv NC_RECV 1477669436.269588
time= 0.002319
Sync Start sm_w/read_S78 _Send NC_SEND 1477669436.289302
Sync End sm_w/read_S78 _Send NC_SEND 1477669436.289403
time= 0.000101

＆＃13;

TensorFlow通信时间不相等

0 个答案: