Question

我目前正在尝试修改使用 Akka Actors 实施的简单 DistBelief 框架的源代码。原始源代码位于：http://alexminnaar.com/implementing-the-distbelief-deep-neural-network-training-framework-with-akka.html。原始实现仅基于Akka Actors，但我想将其扩展到分布式模式。我认为 Akka-Cluster-Sharding 是此任务的正确选项。但我想知道在receive（）方法或者extractShardId（）＆amp;中正确处理传入消息的位置。在actor类中的extractEntityId（）（例如对于 ParameterShard Actor ，您可以在上面给出的链接中看到完整的源代码）。 Akka的官方文档说：* extractEntityId和extractShardId是两个特定于应用程序的函数，用于从传入消息中提取实体标识符和分片标识符。

object ParameterShard {
  case class ParameterRequest(dataShardId: Int, layerId: Int)
  case class LatestParameters(weights: DenseMatrix[Double])
}

class ParamServer(shardId: Int,
                  numberOfShards: Int,
                  learningRate: Double,
                  initialWeight: LayerWeight) extends Actor with ActorLogging {


  val shardName: String = "ParamServer"

  val extractEntityId: ShardRegion.ExtractEntityId = {
      //case ps: ParameterRequest => (ps.dataShardId.toString, ps)

  }

  val extractShardId: ShardRegion.ExtractShardId = {
      //case ps: ParameterRequest => (ps.dataShardId % numberOfShards).toString
  }
  //weights initialize randomly
  var latestParameter: LayerWeight = initialWeight

  def receive = {

    //A layer corresponding to this shardId in some model replica has requested the latest version of the parameters.
    case ParameterRequest(shardId, layerId) => {
      log.info(s"layer ${layerId} weights read by model replica ${shardId}")
      context.sender() ! LatestParameters(latestParameter)
    }

    /*
    A layer corresponding to this shardId in some model replica has computed a gradient, so we must update our
    parameters according to this gradient.
    */
    case Gradient(g, replicaId, layerId) => {
      log.info(s"layer ${layerId} weights updated by model replica ${replicaId}")
      latestParameter = latestParameter + g.t * learningRate
    }

  }

}

Answer 1

我对DistBelief一无所知，所以我无法确定哪个演员被识别出来。所以，对不起，我无法提供示例实现。但是让我们举一个例子：你想从不同的域中获取/示例url。所以你的演员看起来像

class FetchActor(domain: String) extends Actor {
  def receive = {
    case Fetch => // do the job
  }
}

object FetchActor {
  case object Fetch
}

为了对该actor使用集群分片，请像这样重构

class FetchActor extends Actor {
  def receive = {
    case Fetch(domain) => // do the job
  }
}

object FetchActor {
  case class Fetch(domain: String) // move parameter to message
  val entityIdExtractor = {
    case msg @ Fetch(domain) => (domain, msg)   // Which message FetchActor process
  }
  val shardIdExtractor = {
    case Fetch(domain) => domain   // What is shard id
  }
}

我决定在该示例中将整个域用作shardIdExtractor，但您的应用程序可能需要顶级域。

那就够了。只需使用群集分片启动FetchActor，akka就会自动处理它的分发。

Akka-Cluster-Sharding：receive（）vs extarctShardId（）＆amp; extractEntityId（）

1 个答案: