如何使用horovod对正常值进行allreduce操作？

Question

Horovod可以很容易地计算张量流的梯度。我们可以使用horovod来计算普通值吗？例如：

import horovod.tensorflow as hvd
import numpy as np
hvd.init()
hvd_r=int(hvd.rank())
#each process compute a small part of something and then compute the average etc.
test_array=np.random.rand(100,100,100)
#compute a small part
x=np.mean(test_array[hvd_r*10:(hvd_r+1)*(10),:,:])
#compute the average for all processes
y=hvd.sth(x)
#only one process print out the result
if(hvd_r==0):
    print("mean of the big array is %f"%y)

Answer 1

import horovod.keras as hvd
import numpy as np
hvd.init()
hvd_r=int(hvd.rank())
#each process compute a small part of something and then compute the average etc.
test_array=np.random.rand(100,100,100)
#compute a small part
x=np.mean(test_array[hvd_r*10:(hvd_r+1)*(10),:,:])
#compute the average for all processes
y=hvd.allreduce(x)
#only one process print out the result
if(hvd_r==0):
    print("mean of the big array is %f"%y)

Answer 2

在第10行中，将其更改为

y=hvd.allreduce(x)

它将给您想要的结果。